2016-05-17 20:45:42,641 INFO platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(197)) - found 1 test platforms from classpath
2016-05-17 20:45:42,663 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(246)) - adding test: cascading.BasicPipesPlatformTest, with platform: hadoop2-tez
2016-05-17 20:45:42,688 INFO tez.Hadoop2TezPlatform (Hadoop2TezPlatform.java:setUp(136)) - not using cluster
2016-05-17 20:45:42,999 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-05-17 20:45:43,341 INFO platform.TestPlatform (TestPlatform.java:getGlobalProperties(75)) - platform property overrides:
2016-05-17 20:45:43,568 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:43,570 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:43,695 INFO property.AppProps (AppProps.java:getAppID(177)) - using app.id: 0FD517CFF093459A99073014D46E7D06
2016-05-17 20:45:43,898 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.091
2016-05-17 20:45:43,935 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.129
2016-05-17 20:45:43,937 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-17 20:45:43,937 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-17 20:45:43,939 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:44,086 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 20:45:44,261 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain
2016-05-17 20:45:44,490 INFO util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 3.1.0
2016-05-17 20:45:44,493 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 20:45:44,494 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:44,495 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain"]
2016-05-17 20:45:44,495 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 20:45:44,495 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 20:45:44,497 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 20:45:44,498 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...stSimpleChain/simplechain
2016-05-17 20:45:44,499 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:44,516 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 20:45:44,668 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:44,674 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:44,674 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:44,715 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4/.tez/application_1463517944674_0001 doesn't exist and is created
2016-05-17 20:45:44,773 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:44,773 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:44,775 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4/.tez/application_1463517944674_0001
2016-05-17 20:45:44,806 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517944674_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:44,846 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:44,862 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:44,875 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: NEW
2016-05-17 20:45:44,875 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(238)) - DAGAppMaster is not started wait for 100ms...
2016-05-17 20:45:44,875 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:44,888 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:44,892 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:44,895 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4/.tez/application_1463517944674_0001/localmode-log-dir
2016-05-17 20:45:44,896 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4/.tez/application_1463517944674_0001/localmode-log-dir/history.txt.appattempt_1463517944674_0001_000000, maxErrors=10
2016-05-17 20:45:44,896 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:44,899 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517944674_0001_000000, appSubmitTime=1463517944792, launchTime=1463517944805
2016-05-17 20:45:44,909 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:44,909 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:44,914 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:44,916 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:44,976 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-05-17 20:45:44,976 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-05-17 20:45:44,976 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:44,980 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:41451
2016-05-17 20:45:45,022 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:45,077 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-05-17 20:45:45,077 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-05-17 20:45:45,077 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:45,090 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:45,091 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517944674_0001_000000, startTime=1463517945090
2016-05-17 20:45:45,091 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:45,177 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:45,194 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:45,195 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...stSimpleChain/simplechain, applicationId=application_1463517944674_0001, dagName=[52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4] test/(1/1) ...stSimpleChain/simplechain
2016-05-17 20:45:45,418 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4] test/(1/1) ...stSimpleChain/simplechain
2016-05-17 20:45:45,525 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517944674_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4/.tez/application_1463517944674_0001/localmode-log-dir/dag_1463517944674_0001_1.dot
2016-05-17 20:45:45,530 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4/.tez/application_1463517944674_0001/localmode-log-dir/dag_1463517944674_0001_1-tez-dag.pb.txt
2016-05-17 20:45:45,627 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517944674_0001_1
2016-05-17 20:45:45,627 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:45,627 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4] test/(1/1) ...stSimpleChain/simplechain
2016-05-17 20:45:45 Running Dag: dag_1463517944674_0001_1
2016-05-17 20:45:45,672 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:45,672 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517944674_0001_1, submitTime=1463517945418
2016-05-17 20:45:45,706 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DA216359D1C94EB2B0C8F756359FEFD6
2016-05-17 20:45:45,715 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:45,717 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517944674_0001_1, initTime=1463517945674
2016-05-17 20:45:45,718 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517944674_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:45,718 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:45,720 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:DAG_STARTED]: dagID=dag_1463517944674_0001_1, startTime=1463517945719
2016-05-17 20:45:45,720 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:45,721 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:45,722 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517944674_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:45,723 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B7CF60ABA0644A7FA70974D7766572AB : {7EDF6B7FD6664EBA8177ECC7A9D7A4EB={InputName=7EDF6B7FD6664EBA8177ECC7A9D7A4EB}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:45,723 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 7EDF6B7FD6664EBA8177ECC7A9D7A4EB, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:45,723 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB]
2016-05-17 20:45:45,730 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB]
2016-05-17 20:45:45,730 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB]
2016-05-17 20:45:45,733 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB]. Starting root input initializers: 1
2016-05-17 20:45:45,736 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...stSimpleChain/simplechain, applicationId=application_1463517944674_0001, dagName=[52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4] test/(1/1) ...stSimpleChain/simplechain
2016-05-17 20:45:45,773 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 7EDF6B7FD6664EBA8177ECC7A9D7A4EB on vertex vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB]
2016-05-17 20:45:45,792 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:45,795 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:45,797 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 7EDF6B7FD6664EBA8177ECC7A9D7A4EB asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:45,799 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:45,799 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:45,802 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90]
2016-05-17 20:45:45,831 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 20:45:45,831 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90]
2016-05-17 20:45:45,845 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 20:45:45,848 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 7EDF6B7FD6664EBA8177ECC7A9D7A4EB on vertex vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB]
2016-05-17 20:45:45,849 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90]
2016-05-17 20:45:45,852 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:45,852 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B99DE4A7E23B4D85A8EE364452481C90, vertexId=vertex_1463517944674_0001_1_01, initRequestedTime=1463517945792, initedTime=1463517945851, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 20:45:45,853 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90] transitioned from NEW to INITED due to event V_INIT
2016-05-17 20:45:45,853 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:45,853 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {7EDF6B7FD6664EBA8177ECC7A9D7A4EB=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:45,854 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB] parallelism set to 1
2016-05-17 20:45:45,854 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:45,860 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463517944674_0001, with dag id: dag_1463517944674_0001_1
2016-05-17 20:45:45,869 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517944674_0001_1_00, eventCount=1
2016-05-17 20:45:45,870 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:45,870 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B7CF60ABA0644A7FA70974D7766572AB, vertexId=vertex_1463517944674_0001_1_00, initRequestedTime=1463517945723, initedTime=1463517945870, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:45,871 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB] with distanceFromRoot: 0
2016-05-17 20:45:45,872 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:45,876 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6]
2016-05-17 20:45:45,894 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 20:45:45,894 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6]
2016-05-17 20:45:45,895 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6]
2016-05-17 20:45:45,895 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6]
2016-05-17 20:45:45,896 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=A7B573490F9540059E3AEE82DB4B0EF9, vertexId=vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:45,902 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=A7B573490F9540059E3AEE82DB4B0EF9, vertexId=vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6]
2016-05-17 20:45:45,920 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=A7B573490F9540059E3AEE82DB4B0EF9, vertexName=DA216359D1C94EB2B0C8F756359FEFD6, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:45,938 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=A7B573490F9540059E3AEE82DB4B0EF9, vertexId=vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6]
2016-05-17 20:45:45,942 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:45,942 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DA216359D1C94EB2B0C8F756359FEFD6, vertexId=vertex_1463517944674_0001_1_02, initRequestedTime=1463517945876, initedTime=1463517945942, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 20:45:45,943 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6] transitioned from NEW to INITED due to event V_INIT
2016-05-17 20:45:45,944 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B7CF60ABA0644A7FA70974D7766572AB
2016-05-17 20:45:45,949 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB]
2016-05-17 20:45:45,951 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:45,951 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517944674_0001_1_00, startRequestedTime=1463517945944, startedTime=1463517945944
2016-05-17 20:45:45,953 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:45,981 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:45,981 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:TASK_STARTED]: vertexName=B7CF60ABA0644A7FA70974D7766572AB, taskId=task_1463517944674_0001_1_00_000000, scheduledTime=1463517945981, launchTime=1463517945981
2016-05-17 20:45:45,982 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517944674_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:45,982 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463517944674_0001_1_00 for vertex: vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90] numStartedSources: 1 numSources: 1
2016-05-17 20:45:45,983 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90] with distanceFromRoot: 1
2016-05-17 20:45:45,985 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB]
2016-05-17 20:45:45,989 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B7CF60ABA0644A7FA70974D7766572AB in vertex: B99DE4A7E23B4D85A8EE364452481C90
2016-05-17 20:45:45,992 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B99DE4A7E23B4D85A8EE364452481C90 with 1 source tasks and 1 pending tasks
2016-05-17 20:45:45,992 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:45,992 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517944674_0001_1_01, startRequestedTime=1463517945986, startedTime=1463517945986
2016-05-17 20:45:45,993 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:45,995 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4] test/(1/1) ...stSimpleChain/simplechain, VertexName: B7CF60ABA0644A7FA70974D7766572AB, VertexParallelism: 1, TaskAttemptID:attempt_1463517944674_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7EDF6B7FD6664EBA8177ECC7A9D7A4EB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B99DE4A7E23B4D85A8EE364452481C90, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:45,997 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517944674_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:45,997 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463517944674_0001_1_01 for vertex: vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6] numStartedSources: 1 numSources: 1
2016-05-17 20:45:45,998 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6] with distanceFromRoot: 2
2016-05-17 20:45:45,998 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:45,998 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B99DE4A7E23B4D85A8EE364452481C90 in vertex: DA216359D1C94EB2B0C8F756359FEFD6
2016-05-17 20:45:45,999 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: DA216359D1C94EB2B0C8F756359FEFD6 with 1 source tasks and 1 pending tasks
2016-05-17 20:45:45,999 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:46,000 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517944674_0001_1_02, startRequestedTime=1463517945998, startedTime=1463517945998
2016-05-17 20:45:46,000 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:46,038 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:46,042 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:46,042 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:46,046 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517944674_0001_00_000001
2016-05-17 20:45:46,046 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:46,052 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517944674_0001_00_000001
2016-05-17 20:45:46,052 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:46,057 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:46,058 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:46,059 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517944674_0001_00_000001, launchTime=1463517946058
2016-05-17 20:45:46,062 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517944674_0001_00_000001
2016-05-17 20:45:46,068 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517944674_0001_1_00_000000_0] to container: [container_1463517944674_0001_00_000001]
2016-05-17 20:45:46,068 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:46,071 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517944674_0001_1_00_000000_0] started. Is using containerId: [container_1463517944674_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:46,071 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517944674_0001_00_000001 given task: attempt_1463517944674_0001_1_00_000000_0
2016-05-17 20:45:46,072 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517944674_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517944674_0001_1_00_000000_0
2016-05-17 20:45:46,072 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:46,072 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B7CF60ABA0644A7FA70974D7766572AB, taskAttemptId=attempt_1463517944674_0001_1_00_000000_0, startTime=1463517946069, containerId=container_1463517944674_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517944674_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:46,077 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517944674_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:46,081 INFO counters.Limits (Limits.java:ensureInitialized(59)) - Counter limits initialized with parameters: GROUP_NAME_MAX=128, MAX_GROUPS=500, COUNTER_NAME_MAX=64, MAX_COUNTERS=1200
2016-05-17 20:45:46 Starting to run new task attempt: attempt_1463517944674_0001_1_00_000000_0
2016-05-17 20:45:46,082 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517944674_0001_1_00_000000_0
2016-05-17 20:45:46,082 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:46,083 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:46,084 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:46,084 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:46,086 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:46,089 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517944674_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:46,121 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:46,122 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4] test/(1/1) ...stSimpleChain/simplechain, VertexName: B7CF60ABA0644A7FA70974D7766572AB, VertexParallelism: 1, TaskAttemptID:attempt_1463517944674_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7EDF6B7FD6664EBA8177ECC7A9D7A4EB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B99DE4A7E23B4D85A8EE364452481C90, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:46,124 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:46,129 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517944674_0001_1_00_000000_0
2016-05-17 20:45:46,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:46,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:46,131 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517944674_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:46,137 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=7EDF6B7FD6664EBA8177ECC7A9D7A4EB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:46,138 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:46,139 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B99DE4A7E23B4D85A8EE364452481C90, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 20:45:46,141 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:46,143 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:46,144 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 7EDF6B7FD6664EBA8177ECC7A9D7A4EB
2016-05-17 20:45:46,153 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 7EDF6B7FD6664EBA8177ECC7A9D7A4EB
2016-05-17 20:45:46,155 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B99DE4A7E23B4D85A8EE364452481C90
2016-05-17 20:45:46,167 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 20:45:46,168 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B99DE4A7E23B4D85A8EE364452481C90
2016-05-17 20:45:46,168 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B99DE4A7E23B4D85A8EE364452481C90
2016-05-17 20:45:46,171 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:46,173 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:46,174 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 7EDF6B7FD6664EBA8177ECC7A9D7A4EB
2016-05-17 20:45:46,175 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 7EDF6B7FD6664EBA8177ECC7A9D7A4EB
2016-05-17 20:45:46,186 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B7CF60ABA0644A7FA70974D7766572AB, ordinal: 0
2016-05-17 20:45:46,186 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B7CF60ABA0644A7FA70974D7766572AB, mem on start (mb), free: 228, total: 281, max: 672
2016-05-17 20:45:46,186 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:46,187 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:46,187 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:46,188 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:46,190 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:46,195 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:46,196 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:46,196 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 20:45:46,197 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:46,198 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 20:45:46,198 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 7EDF6B7FD6664EBA8177ECC7A9D7A4EB, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:46,198 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B99DE4A7E23B4D85A8EE364452481C90, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 20:45:46,199 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:46,201 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 7EDF6B7FD6664EBA8177ECC7A9D7A4EB being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:46,201 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:46,203 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:46,201 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 7EDF6B7FD6664EBA8177ECC7A9D7A4EB
2016-05-17 20:45:46,204 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:46,205 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 7EDF6B7FD6664EBA8177ECC7A9D7A4EB
2016-05-17 20:45:46,205 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:46,209 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517944674_0001_1_00_000000_0
2016-05-17 20:45:46,210 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:46,220 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@75dafc0b
2016-05-17 20:45:46,233 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@75dafc0b
2016-05-17 20:45:46,236 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:46,236 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:46,248 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:46,278 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 7EDF6B7FD6664EBA8177ECC7A9D7A4EB
2016-05-17 20:45:46,278 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: F5ADF73F391F48779399EEE33FA9E9C0
2016-05-17 20:45:46,279 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] F5ADF73F391F48779399EEE33FA9E9C0
2016-05-17 20:45:46,324 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@3676fc91; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@3dbfaf86; comparator=cascading.tuple.hadoop.util.TupleComparator@8dc2faa; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 20:45:46,329 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 20:45:46,332 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 20:45:46,332 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:46,445 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 20:45:46,445 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 20:45:46,445 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 20:45:46,446 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 20:45:46,446 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 20:45:46,449 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:46,449 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:46,451 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B7CF60ABA0644A7FA70974D7766572AB, all 1 inputs ready in: 00:00:00.001
2016-05-17 20:45:46,459 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517944674_0001_1_00_000000_0
2016-05-17 20:45:46,459 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B7CF60ABA0644A7FA70974D7766572AB, mem on close (mb), free: 116, total: 281, max: 672
2016-05-17 20:45:46,460 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 20:45:46,460 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 20:45:46,460 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 210; bufvoid = 104857600
2016-05-17 20:45:46,460 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214360(104857440); length = 37/6553600
2016-05-17 20:45:46,478 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 20:45:46 Completed running task attempt: attempt_1463517944674_0001_1_00_000000_0
2016-05-17 20:45:46,498 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517944674_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:46,499 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517944674_0001_1_00, eventCount=1
2016-05-17 20:45:46,503 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463517946451, Process_Duration=8, Process_End_Time=1463517946459, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 20:45:46,503 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:46,504 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:46,504 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:46,507 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517944674_0001_00_000001
2016-05-17 20:45:46,507 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B7CF60ABA0644A7FA70974D7766572AB, taskAttemptId=attempt_1463517944674_0001_1_00_000000_0, startTime=1463517946069, finishTime=1463517946500, timeTaken=431, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517946451, Process_Duration=8, Process_End_Time=1463517946459, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 20:45:46,508 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517944674_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:46,508 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:46,512 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517944674_0001_1_00_000000_0
2016-05-17 20:45:46,514 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:46,517 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:TASK_FINISHED]: vertexName=B7CF60ABA0644A7FA70974D7766572AB, taskId=task_1463517944674_0001_1_00_000000, startTime=1463517946069, finishTime=1463517946513, timeTaken=444, status=SUCCEEDED, successfulAttemptID=attempt_1463517944674_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517946451, Process_Duration=8, Process_End_Time=1463517946459, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 20:45:46,520 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517944674_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:46,520 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:46,524 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB] : 1
2016-05-17 20:45:46,524 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB]
2016-05-17 20:45:46,541 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:46,545 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:VERTEX_FINISHED]: vertexName=B7CF60ABA0644A7FA70974D7766572AB, vertexId=vertex_1463517944674_0001_1_00, initRequestedTime=1463517945723, initedTime=1463517945870, startRequestedTime=1463517945944, startedTime=1463517945944, finishTime=1463517946525, timeTaken=581, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517946451, Process_Duration=8, Process_End_Time=1463517946459, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463517946069, firstTasksToStart=[ task_1463517944674_0001_1_00_000000 ], lastTaskFinishTime=1463517946500, lastTasksToFinish=[ task_1463517944674_0001_1_00_000000 ], minTaskDuration=431, maxTaskDuration=431, avgTaskDuration=431.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517944674_0001_1_00_000000 ], longestDurationTasks=[ task_1463517944674_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:46,546 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:46,547 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:46,548 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517944674_0001_00_000001, stoppedTime=1463517946547, exitStatus=0
2016-05-17 20:45:46,549 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:46,550 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90] attempt: attempt_1463517944674_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 20:45:46,550 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B99DE4A7E23B4D85A8EE364452481C90
2016-05-17 20:45:46,551 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90]
2016-05-17 20:45:46,553 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 20:45:46,554 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463517944674_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 20:45:46,554 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:46,554 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:TASK_STARTED]: vertexName=B99DE4A7E23B4D85A8EE364452481C90, taskId=task_1463517944674_0001_1_01_000000, scheduledTime=1463517946554, launchTime=1463517946554
2016-05-17 20:45:46,555 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517944674_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:46,555 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90]
2016-05-17 20:45:46,556 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 20:45:46,557 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4] test/(1/1) ...stSimpleChain/simplechain, VertexName: B99DE4A7E23B4D85A8EE364452481C90, VertexParallelism: 1, TaskAttemptID:attempt_1463517944674_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B7CF60ABA0644A7FA70974D7766572AB, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=DA216359D1C94EB2B0C8F756359FEFD6, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:46,557 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517944674_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:46,558 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:46,561 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517944674_0001_00_000002
2016-05-17 20:45:46,563 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:46,566 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517944674_0001_00_000002
2016-05-17 20:45:46,566 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:46,567 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:46,568 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517944674_0001_00_000002, launchTime=1463517946567
2016-05-17 20:45:46,569 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:46,569 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517944674_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:46,569 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517944674_0001_00_000001: 62 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:46,569 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517944674_0001_00_000001, Exiting
2016-05-17 20:45:46,570 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517944674_0001_00_000001
2016-05-17 20:45:46,570 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517944674_0001_00_000001
2016-05-17 20:45:46,569 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517944674_0001_00_000002
2016-05-17 20:45:46,573 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517944674_0001_1_01_000000_0] to container: [container_1463517944674_0001_00_000002]
2016-05-17 20:45:46,573 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:46,574 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517944674_0001_00_000002 given task: attempt_1463517944674_0001_1_01_000000_0
2016-05-17 20:45:46,574 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517944674_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517944674_0001_1_01_000000_0
2016-05-17 20:45:46 Starting to run new task attempt: attempt_1463517944674_0001_1_01_000000_0
2016-05-17 20:45:46,575 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517944674_0001_1_01_000000_0] started. Is using containerId: [container_1463517944674_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 20:45:46,576 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517944674_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:46,577 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:46,577 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517944674_0001_00_000001 completed successfully
2016-05-17 20:45:46,575 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517944674_0001_1_01_000000_0
2016-05-17 20:45:46,577 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B99DE4A7E23B4D85A8EE364452481C90, taskAttemptId=attempt_1463517944674_0001_1_01_000000_0, startTime=1463517946574, containerId=container_1463517944674_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517944674_0001_00_000002/teamcity, completedLogs=
2016-05-17 20:45:46,577 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:46,579 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:46,579 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:46,579 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:46,579 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:46,579 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:46,580 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4] test/(1/1) ...stSimpleChain/simplechain, VertexName: B99DE4A7E23B4D85A8EE364452481C90, VertexParallelism: 1, TaskAttemptID:attempt_1463517944674_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B7CF60ABA0644A7FA70974D7766572AB, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=DA216359D1C94EB2B0C8F756359FEFD6, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:46,580 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:46,586 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517944674_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:46,591 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517944674_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 20:45:46,592 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 20:45:46,593 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517944674_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:46,586 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517944674_0001_1_01_000000_0
2016-05-17 20:45:46,594 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:46,594 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:46,596 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B7CF60ABA0644A7FA70974D7766572AB, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 20:45:46,597 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:46,599 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B7CF60ABA0644A7FA70974D7766572AB
2016-05-17 20:45:46,602 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:46,603 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DA216359D1C94EB2B0C8F756359FEFD6, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 20:45:46,604 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:46,604 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DA216359D1C94EB2B0C8F756359FEFD6
2016-05-17 20:45:46,608 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:46,616 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 20:45:46,617 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: DA216359D1C94EB2B0C8F756359FEFD6
2016-05-17 20:45:46,617 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DA216359D1C94EB2B0C8F756359FEFD6
2016-05-17 20:45:46,627 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 20:45:46,627 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 20:45:46,628 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B7CF60ABA0644A7FA70974D7766572AB
2016-05-17 20:45:46,629 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B7CF60ABA0644A7FA70974D7766572AB
2016-05-17 20:45:46,633 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B99DE4A7E23B4D85A8EE364452481C90, ordinal: 1
2016-05-17 20:45:46,633 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B99DE4A7E23B4D85A8EE364452481C90, mem on start (mb), free: 105, total: 281, max: 672
2016-05-17 20:45:46,633 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:46,633 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:46,633 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:46,634 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:46,634 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:46,634 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 20:45:46,638 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 20:45:46,639 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:46,639 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 24, TotalRequested: 739036352, TotalRequestedScaled: 3.69518176E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:1.05
2016-05-17 20:45:46,640 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DA216359D1C94EB2B0C8F756359FEFD6, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=66985083
2016-05-17 20:45:46,640 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B7CF60ABA0644A7FA70974D7766572AB, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=405125774
2016-05-17 20:45:46,640 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:46,641 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B7CF60ABA0644A7FA70974D7766572AB
2016-05-17 20:45:46,643 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B7CF60ABA0644A7FA70974D7766572AB being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:46,647 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:46,648 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:46,652 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:46,653 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 20:45:46,656 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B7CF60ABA0644A7FA70974D7766572AB with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 20:45:46,658 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=405125774. Updated to: ShuffleMem=405125774, postMergeMem: 0
2016-05-17 20:45:46,658 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=405125774, maxSingleShuffleLimit=101281440, mergeThreshold=364613184, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 20:45:46,662 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 20:45:46,666 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B7CF60ABA0644A7FA70974D7766572AB
2016-05-17 20:45:46,666 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:46,667 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517944674_0001_1_01_000000_0
2016-05-17 20:45:46,669 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463517944674_0001_1_00_000000_0_10002, runDuration: 326337, ]
2016-05-17 20:45:46,672 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-05-17 20:45:46,678 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463517944674_0001_1_00_000000_0_10002] to DISK_DIRECT, CompressedSize=204, DecompressedSize=200,EndTime=1463517946678, TimeTaken=2, Rate=0.10 MB/s
2016-05-17 20:45:46,679 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B7CF60ABA0644A7FA70974D7766572AB
2016-05-17 20:45:46,680 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: F5ADF73F391F48779399EEE33FA9E9C0
2016-05-17 20:45:46,681 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: 9EE434D1716D4BCC80D15007F33A09C4
2016-05-17 20:45:46,681 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 20:45:46,681 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [B7CF60ABA0644A7FA70974D7766572AB] #1 in 6ms
2016-05-17 20:45:46,681 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] 9EE434D1716D4BCC80D15007F33A09C4
2016-05-17 20:45:46,684 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@3a6298b2; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@1f16ec10; comparator=cascading.tuple.hadoop.util.TupleComparator@19e41e62; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 20:45:46,685 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 20:45:46,685 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 20:45:46,685 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:46,685 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B7CF60ABA0644A7FA70974D7766572AB] #1, status:false, isInterrupted:false
2016-05-17 20:45:46,748 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 16515068(66060272)
2016-05-17 20:45:46,751 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 63
2016-05-17 20:45:46,751 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 20:45:46,751 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 52848232
2016-05-17 20:45:46,753 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 66060288
2016-05-17 20:45:46,753 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 16515068; length = 4128768
2016-05-17 20:45:46,754 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] F5ADF73F391F48779399EEE33FA9E9C0, for 1 inputs
2016-05-17 20:45:46,757 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 204 bytes from disk
2016-05-17 20:45:46,759 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:46,763 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 20:45:46,768 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 175 bytes
2016-05-17 20:45:46,769 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B7CF60ABA0644A7FA70974D7766572AB
2016-05-17 20:45:46,770 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B99DE4A7E23B4D85A8EE364452481C90, all 1 inputs ready in: 00:00:00.015
2016-05-17 20:45:46,773 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 20:45:46,775 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@73778636; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@2c0c7e73
2016-05-17 20:45:46,786 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517944674_0001_1_01_000000_0
2016-05-17 20:45:46,786 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B99DE4A7E23B4D85A8EE364452481C90, mem on close (mb), free: 99, total: 345, max: 672
2016-05-17 20:45:46,786 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B7CF60ABA0644A7FA70974D7766572AB
2016-05-17 20:45:46,787 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 20:45:46,788 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 20:45:46,788 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:46,789 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 20:45:46,789 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 20:45:46,790 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 20:45:46,791 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 20:45:46,791 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 184; bufvoid = 66060288
2016-05-17 20:45:46,791 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 16515068(66060272); kvend = 16515040(66060160); length = 29/4128768
2016-05-17 20:45:46,796 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 20:45:46 Completed running task attempt: attempt_1463517944674_0001_1_01_000000_0
2016-05-17 20:45:46,802 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517944674_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:46,803 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517944674_0001_1_01, eventCount=1
2016-05-17 20:45:46,804 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:46,809 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B99DE4A7E23B4D85A8EE364452481C90, taskAttemptId=attempt_1463517944674_0001_1_01_000000_0, startTime=1463517946574, finishTime=1463517946803, timeTaken=229, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 49, File System Counters, FILE_BYTES_READ=268, FILE_BYTES_WRITTEN=242, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=28, MERGE_PHASE_TIME=116, FIRST_EVENT_RECEIVED=22, LAST_EVENT_RECEIVED=22, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517946770, Process_Duration=16, Process_End_Time=1463517946786, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0
2016-05-17 20:45:46,809 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517944674_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:46,809 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517944674_0001_1_01_000000_0
2016-05-17 20:45:46,810 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:46,811 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:46,813 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:TASK_FINISHED]: vertexName=B99DE4A7E23B4D85A8EE364452481C90, taskId=task_1463517944674_0001_1_01_000000, startTime=1463517946574, finishTime=1463517946809, timeTaken=235, status=SUCCEEDED, successfulAttemptID=attempt_1463517944674_0001_1_01_000000_0, diagnostics=, counters=Counters: 49, File System Counters, FILE_BYTES_READ=268, FILE_BYTES_WRITTEN=242, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=28, MERGE_PHASE_TIME=116, FIRST_EVENT_RECEIVED=22, LAST_EVENT_RECEIVED=22, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517946770, Process_Duration=16, Process_End_Time=1463517946786, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0
2016-05-17 20:45:46,813 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517944674_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:46,814 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90] : 1
2016-05-17 20:45:46,814 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90]
2016-05-17 20:45:46,815 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:46,816 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 49 [[File System Counters FILE_BYTES_READ=268, FILE_BYTES_WRITTEN=242, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=28, MERGE_PHASE_TIME=116, FIRST_EVENT_RECEIVED=22, LAST_EVENT_RECEIVED=22][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463517946770, Process_Duration=16, Process_End_Time=1463517946786, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0]]
2016-05-17 20:45:46,819 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:46,819 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:46,821 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:VERTEX_FINISHED]: vertexName=B99DE4A7E23B4D85A8EE364452481C90, vertexId=vertex_1463517944674_0001_1_01, initRequestedTime=1463517945792, initedTime=1463517945851, startRequestedTime=1463517945986, startedTime=1463517945986, finishTime=1463517946814, timeTaken=828, status=SUCCEEDED, diagnostics=, counters=Counters: 49, File System Counters, FILE_BYTES_READ=268, FILE_BYTES_WRITTEN=242, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=28, MERGE_PHASE_TIME=116, FIRST_EVENT_RECEIVED=22, LAST_EVENT_RECEIVED=22, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517946770, Process_Duration=16, Process_End_Time=1463517946786, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, vertexStats=firstTaskStartTime=1463517946574, firstTasksToStart=[ task_1463517944674_0001_1_01_000000 ], lastTaskFinishTime=1463517946803, lastTasksToFinish=[ task_1463517944674_0001_1_01_000000 ], minTaskDuration=229, maxTaskDuration=229, avgTaskDuration=229.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517944674_0001_1_01_000000 ], longestDurationTasks=[ task_1463517944674_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:46,823 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:46,823 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:46,822 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517944674_0001_00_000002
2016-05-17 20:45:46,825 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517944674_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 20:45:46,825 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517944674_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:46,826 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517944674_0001_00_000002, Exiting
2016-05-17 20:45:46,826 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517944674_0001_00_000002
2016-05-17 20:45:46,826 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517944674_0001_00_000002
2016-05-17 20:45:46,827 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:46,827 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517944674_0001_00_000002, stoppedTime=1463517946824, exitStatus=0
2016-05-17 20:45:46,828 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:46,828 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517944674_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:46,828 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517944674_0001_00_000002 completed successfully
2016-05-17 20:45:46,828 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6] attempt: attempt_1463517944674_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 20:45:46,829 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: DA216359D1C94EB2B0C8F756359FEFD6
2016-05-17 20:45:46,829 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6]
2016-05-17 20:45:46,830 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 20:45:46,830 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517944674_0001_00_000002
2016-05-17 20:45:46,830 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:46,831 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:46,832 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:TASK_STARTED]: vertexName=DA216359D1C94EB2B0C8F756359FEFD6, taskId=task_1463517944674_0001_1_02_000000, scheduledTime=1463517946831, launchTime=1463517946831
2016-05-17 20:45:46,832 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517944674_0001_1_02_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:46,833 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6]
2016-05-17 20:45:46,833 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4] test/(1/1) ...stSimpleChain/simplechain, VertexName: DA216359D1C94EB2B0C8F756359FEFD6, VertexParallelism: 1, TaskAttemptID:attempt_1463517944674_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B99DE4A7E23B4D85A8EE364452481C90, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A7B573490F9540059E3AEE82DB4B0EF9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:46,834 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517944674_0001_1_02_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:46,834 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:46,835 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517944674_0001_00_000003
2016-05-17 20:45:46,836 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000003 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:46,837 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517944674_0001_00_000003
2016-05-17 20:45:46,838 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 20:45:46,840 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000003 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:46,841 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517944674_0001_00_000003, launchTime=1463517946840
2016-05-17 20:45:46,840 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:46,842 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517944674_0001_00_000003
2016-05-17 20:45:46,843 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517944674_0001_1_02_000000_0] to container: [container_1463517944674_0001_00_000003]
2016-05-17 20:45:46,843 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000003 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:46,843 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517944674_0001_00_000003 given task: attempt_1463517944674_0001_1_02_000000_0
2016-05-17 20:45:46,844 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517944674_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517944674_0001_1_02_000000_0
2016-05-17 20:45:46,844 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517944674_0001_1_02_000000_0] started. Is using containerId: [container_1463517944674_0001_00_000003] on NM: [127.0.0.1:0]
2016-05-17 20:45:46,845 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:46,845 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DA216359D1C94EB2B0C8F756359FEFD6, taskAttemptId=attempt_1463517944674_0001_1_02_000000_0, startTime=1463517946844, containerId=container_1463517944674_0001_00_000003, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517944674_0001_00_000003/teamcity, completedLogs=
2016-05-17 20:45:46,846 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517944674_0001_1_02_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:46,846 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517944674_0001_1_02_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:46 Starting to run new task attempt: attempt_1463517944674_0001_1_02_000000_0
2016-05-17 20:45:46,847 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517944674_0001_1_02_000000_0
2016-05-17 20:45:46,847 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:46,848 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:46,848 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:46,848 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:46,849 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:46,849 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:46,849 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4] test/(1/1) ...stSimpleChain/simplechain, VertexName: DA216359D1C94EB2B0C8F756359FEFD6, VertexParallelism: 1, TaskAttemptID:attempt_1463517944674_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B99DE4A7E23B4D85A8EE364452481C90, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A7B573490F9540059E3AEE82DB4B0EF9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:46,849 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:46,850 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517944674_0001_1_02_000000_0
2016-05-17 20:45:46,855 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:46,855 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:46,859 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:46,859 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B99DE4A7E23B4D85A8EE364452481C90, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 20:45:46,860 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:46,860 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B99DE4A7E23B4D85A8EE364452481C90
2016-05-17 20:45:46,860 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A7B573490F9540059E3AEE82DB4B0EF9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:46,863 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 20:45:46,863 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 20:45:46,864 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B99DE4A7E23B4D85A8EE364452481C90
2016-05-17 20:45:46,865 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B99DE4A7E23B4D85A8EE364452481C90
2016-05-17 20:45:46,863 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:46,866 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A7B573490F9540059E3AEE82DB4B0EF9
2016-05-17 20:45:46,866 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:46,866 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: A7B573490F9540059E3AEE82DB4B0EF9
2016-05-17 20:45:46,865 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:46,864 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517944674_0001_1_02_000000_0 sent events: (0-1)
2016-05-17 20:45:46,882 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DA216359D1C94EB2B0C8F756359FEFD6, ordinal: 2
2016-05-17 20:45:46,883 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DA216359D1C94EB2B0C8F756359FEFD6, mem on start (mb), free: 93, total: 345, max: 672
2016-05-17 20:45:46,883 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:46,884 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:46,885 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:46,891 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:46,892 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A7B573490F9540059E3AEE82DB4B0EF9
2016-05-17 20:45:46,892 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:46,893 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:46,893 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 20:45:46,894 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:46,894 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:46,895 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:46,895 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 20:45:46,896 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B99DE4A7E23B4D85A8EE364452481C90, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 20:45:46,897 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A7B573490F9540059E3AEE82DB4B0EF9, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:46,897 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:46,898 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B99DE4A7E23B4D85A8EE364452481C90 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:46,898 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B99DE4A7E23B4D85A8EE364452481C90
2016-05-17 20:45:46,899 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:46,900 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:46,901 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:46,901 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 20:45:46,902 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B99DE4A7E23B4D85A8EE364452481C90 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 20:45:46,902 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 20:45:46,902 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 20:45:46,902 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 20:45:46,906 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B99DE4A7E23B4D85A8EE364452481C90
2016-05-17 20:45:46,906 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:46,908 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517944674_0001_1_02_000000_0
2016-05-17 20:45:46,912 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463517944674_0001_1_01_000000_0_10006, runDuration: 195730, ]
2016-05-17 20:45:46,913 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-05-17 20:45:46,914 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463517944674_0001_1_01_000000_0_10006] to DISK_DIRECT, CompressedSize=210, DecompressedSize=206,EndTime=1463517946914, TimeTaken=1, Rate=0.20 MB/s
2016-05-17 20:45:46,915 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B99DE4A7E23B4D85A8EE364452481C90
2016-05-17 20:45:46,916 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 20:45:46,916 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [B99DE4A7E23B4D85A8EE364452481C90] #2 in 3ms
2016-05-17 20:45:46,919 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B99DE4A7E23B4D85A8EE364452481C90] #2, status:false, isInterrupted:false
2016-05-17 20:45:46,919 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: 9EE434D1716D4BCC80D15007F33A09C4
2016-05-17 20:45:46,920 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain"], id: A7B573490F9540059E3AEE82DB4B0EF9
2016-05-17 20:45:46,920 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain"]
2016-05-17 20:45:46,921 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 20:45:46,922 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] 9EE434D1716D4BCC80D15007F33A09C4, for 1 inputs
2016-05-17 20:45:46,924 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 210 bytes from disk
2016-05-17 20:45:46,924 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:46,925 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 20:45:46,926 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 181 bytes
2016-05-17 20:45:46,927 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B99DE4A7E23B4D85A8EE364452481C90
2016-05-17 20:45:46,927 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DA216359D1C94EB2B0C8F756359FEFD6, all 1 inputs ready in: 00:00:00.005
2016-05-17 20:45:46,928 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 20:45:46,930 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@6d1be32; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@341dc81d
2016-05-17 20:45:46,934 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:46,935 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:46,936 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517944674_0001_1_02_000000_0
2016-05-17 20:45:46,937 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517944674_0001_1_02_000000_0 given a go for committing the task output.
2016-05-17 20:45:46,939 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517944674_0001_1_02_000000_0
2016-05-17 20:45:46,939 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DA216359D1C94EB2B0C8F756359FEFD6, mem on close (mb), free: 89, total: 345, max: 672
2016-05-17 20:45:46,940 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B99DE4A7E23B4D85A8EE364452481C90
2016-05-17 20:45:46,941 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 20:45:46,943 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 20:45:46,943 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:46,943 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 20:45:46,944 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 20:45:46 Completed running task attempt: attempt_1463517944674_0001_1_02_000000_0
2016-05-17 20:45:46,946 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517944674_0001_1_02_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:46,948 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=242, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=15, MERGE_PHASE_TIME=26, FIRST_EVENT_RECEIVED=12, LAST_EVENT_RECEIVED=12][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463517946928, Process_Duration=11, Process_End_Time=1463517946939, Read_Duration=2, Tuples_Read=8, Tuples_Written=8, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=8]]
2016-05-17 20:45:46,948 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:46,949 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:46,950 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517944674_0001_00_000003
2016-05-17 20:45:46,951 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DA216359D1C94EB2B0C8F756359FEFD6, taskAttemptId=attempt_1463517944674_0001_1_02_000000_0, startTime=1463517946844, finishTime=1463517946949, timeTaken=105, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=242, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=15, MERGE_PHASE_TIME=26, FIRST_EVENT_RECEIVED=12, LAST_EVENT_RECEIVED=12, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517946928, Process_Duration=11, Process_End_Time=1463517946939, Read_Duration=2, Tuples_Read=8, Tuples_Written=8, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=8
2016-05-17 20:45:46,950 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:46,952 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517944674_0001_1_02_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:46,954 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517944674_0001_1_02_000000_0
2016-05-17 20:45:46,955 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:46,956 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:TASK_FINISHED]: vertexName=DA216359D1C94EB2B0C8F756359FEFD6, taskId=task_1463517944674_0001_1_02_000000, startTime=1463517946844, finishTime=1463517946955, timeTaken=111, status=SUCCEEDED, successfulAttemptID=attempt_1463517944674_0001_1_02_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=242, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=15, MERGE_PHASE_TIME=26, FIRST_EVENT_RECEIVED=12, LAST_EVENT_RECEIVED=12, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517946928, Process_Duration=11, Process_End_Time=1463517946939, Read_Duration=2, Tuples_Read=8, Tuples_Written=8, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=8
2016-05-17 20:45:46,957 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517944674_0001_1_02_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:46,957 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6] : 1
2016-05-17 20:45:46,957 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6]
2016-05-17 20:45:46,958 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:46,960 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:46,963 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:VERTEX_FINISHED]: vertexName=DA216359D1C94EB2B0C8F756359FEFD6, vertexId=vertex_1463517944674_0001_1_02, initRequestedTime=1463517945876, initedTime=1463517945942, startRequestedTime=1463517945998, startedTime=1463517945998, finishTime=1463517946957, timeTaken=959, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=242, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=15, MERGE_PHASE_TIME=26, FIRST_EVENT_RECEIVED=12, LAST_EVENT_RECEIVED=12, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517946928, Process_Duration=11, Process_End_Time=1463517946939, Read_Duration=2, Tuples_Read=8, Tuples_Written=8, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=8, vertexStats=firstTaskStartTime=1463517946844, firstTasksToStart=[ task_1463517944674_0001_1_02_000000 ], lastTaskFinishTime=1463517946949, lastTasksToFinish=[ task_1463517944674_0001_1_02_000000 ], minTaskDuration=105, maxTaskDuration=105, avgTaskDuration=105.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517944674_0001_1_02_000000 ], longestDurationTasks=[ task_1463517944674_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:46,963 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:46,964 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000003 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:46,964 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517944674_0001_00_000003, stoppedTime=1463517946964, exitStatus=0
2016-05-17 20:45:46,965 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000003 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:46,965 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:46,966 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 20:45:46,967 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517944674_0001_1
2016-05-17 20:45:46,968 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517944674_0001_1
2016-05-17 20:45:46,968 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463517944674_0001_1_01 [B99DE4A7E23B4D85A8EE364452481C90]
2016-05-17 20:45:46,969 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: A7B573490F9540059E3AEE82DB4B0EF9 for vertex: vertex_1463517944674_0001_1_02 [DA216359D1C94EB2B0C8F756359FEFD6]
2016-05-17 20:45:46,969 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517944674_0001_00_000003 is valid, but no longer registered, and will be killed
2016-05-17 20:45:46,970 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517944674_0001_00_000003: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:46,970 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517944674_0001_00_000003, Exiting
2016-05-17 20:45:46,970 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517944674_0001_00_000003
2016-05-17 20:45:46,970 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517944674_0001_00_000003
2016-05-17 20:45:46,972 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517944674_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:46,972 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517944674_0001_00_000003 completed successfully
2016-05-17 20:45:46,985 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463517944674_0001_1_00 [B7CF60ABA0644A7FA70974D7766572AB]
2016-05-17 20:45:46,992 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517944674_0001_1, queueSize=0
2016-05-17 20:45:46,992 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:46,993 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517944674_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517944674_0001_1, startTime=1463517945719, finishTime=1463517946985, timeTaken=1266, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2716, FILE_BYTES_WRITTEN=618, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=14, REDUCE_INPUT_RECORDS=18, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=36, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1018691584, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=26, OUTPUT_BYTES=394, OUTPUT_BYTES_WITH_OVERHEAD=406, OUTPUT_BYTES_PHYSICAL=414, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=414, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=414, SHUFFLE_BYTES_DECOMPRESSED=406, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=414, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=43, MERGE_PHASE_TIME=142, FIRST_EVENT_RECEIVED=34, LAST_EVENT_RECEIVED=34, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4390553840149, Process_Duration=35, Process_End_Time=4390553840184, Read_Duration=5, Tuples_Read=28, Tuples_Written=26, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=8
2016-05-17 20:45:46,994 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517944674_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:46,994 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517944674_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:46,995 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517944674_0001_00_000003
2016-05-17 20:45:46,995 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517944674_0001_00_000003 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:46 Completed Dag: dag_1463517944674_0001_1
2016-05-17 20:45:46,996 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517944674_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:46,996 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517944674_0001_1_post
2016-05-17 20:45:46,997 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:46,997 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:47,008 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...stSimpleChain/simplechain, applicationId=application_1463517944674_0001
2016-05-17 20:45:47,013 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:47,013 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:47,014 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:47,014 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:47,014 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:47,015 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:47,018 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:47,019 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:47,019 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:47,019 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:47,020 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:47,021 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:47,021 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:47,027 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/52316B09D1824ABC98EC187C74E4D0A0/AC44FD997E3240528153350EFBFFC6F4/.tez/application_1463517944674_0001
2016-05-17 20:45:47,029 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain/_temporary
2016-05-17 20:45:47,030 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:02.535
2016-05-17 20:45:47,075 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain
2016-05-17 20:45:47,117 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:47,118 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:47,181 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.030
2016-05-17 20:45:47,212 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.060
2016-05-17 20:45:47,214 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:47,214 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:47,214 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:47,233 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-05-17 20:45:47,245 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count
2016-05-17 20:45:47,348 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting
2016-05-17 20:45:47,348 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 20:45:47,348 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count"]
2016-05-17 20:45:47,348 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] parallel execution of steps is enabled: false
2016-05-17 20:45:47,348 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executing total steps: 1
2016-05-17 20:45:47,348 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] allocating management threads: 1
2016-05-17 20:45:47,349 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting step: (1/1) ...splatform/testCount/count
2016-05-17 20:45:47,350 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:47,371 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tez session mode enabled: true
2016-05-17 20:45:47,377 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:47,378 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:47,378 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:47,391 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA/.tez/application_1463517947378_0001 doesn't exist and is created
2016-05-17 20:45:47,410 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:47,410 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:47,412 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA/.tez/application_1463517947378_0001
2016-05-17 20:45:47,416 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517947378_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:47,433 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:47,434 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:47,437 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:47,438 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:47,438 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA/.tez/application_1463517947378_0001/localmode-log-dir
2016-05-17 20:45:47,438 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA/.tez/application_1463517947378_0001/localmode-log-dir/history.txt.appattempt_1463517947378_0001_000000, maxErrors=10
2016-05-17 20:45:47,438 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:47,439 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517947378_0001_000000, appSubmitTime=1463517947416, launchTime=1463517947416
2016-05-17 20:45:47,444 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:47,445 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:47,447 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:47,451 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:52446
2016-05-17 20:45:47,452 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:47,455 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:47,457 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517947378_0001_000000, startTime=1463517947457
2016-05-17 20:45:47,457 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:47,457 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:47,511 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:47,512 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:47,512 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...splatform/testCount/count, applicationId=application_1463517947378_0001, dagName=[79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA] count/(1/1) ...splatform/testCount/count
2016-05-17 20:45:47,533 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA] count/(1/1) ...splatform/testCount/count
2016-05-17 20:45:47,536 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517947378_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA/.tez/application_1463517947378_0001/localmode-log-dir/dag_1463517947378_0001_1.dot
2016-05-17 20:45:47,538 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA/.tez/application_1463517947378_0001/localmode-log-dir/dag_1463517947378_0001_1-tez-dag.pb.txt
2016-05-17 20:45:47,565 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517947378_0001_1
2016-05-17 20:45:47,566 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:47,566 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA] count/(1/1) ...splatform/testCount/count
2016-05-17 20:45:47 Running Dag: dag_1463517947378_0001_1
2016-05-17 20:45:47,582 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517947378_0001_1, submitTime=1463517947533
2016-05-17 20:45:47,582 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:47,583 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex EE2A5BED0ACD4A2DBB81B997AAE33564
2016-05-17 20:45:47,584 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:47,584 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517947378_0001_1, initTime=1463517947582
2016-05-17 20:45:47,584 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:47,585 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517947378_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:47,586 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:DAG_STARTED]: dagID=dag_1463517947378_0001_1, startTime=1463517947585
2016-05-17 20:45:47,586 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:47,586 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:47,586 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517947378_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:47,586 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E07BAAFDDE004DF18A0716FB41FA3255 : {EF1BAF328A81482DAB50C347146DF1C9={InputName=EF1BAF328A81482DAB50C347146DF1C9}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:47,586 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: EF1BAF328A81482DAB50C347146DF1C9, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:47,586 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255]
2016-05-17 20:45:47,587 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255]
2016-05-17 20:45:47,587 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255]
2016-05-17 20:45:47,587 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255]. Starting root input initializers: 1
2016-05-17 20:45:47,588 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:47,588 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564]
2016-05-17 20:45:47,590 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...splatform/testCount/count, applicationId=application_1463517947378_0001, dagName=[79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA] count/(1/1) ...splatform/testCount/count
2016-05-17 20:45:47,594 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 20:45:47,595 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564]
2016-05-17 20:45:47,595 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564]
2016-05-17 20:45:47,595 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564]
2016-05-17 20:45:47,595 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=2363FCD16E1448D4978D6CD5181A13C5, vertexId=vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:47,596 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=2363FCD16E1448D4978D6CD5181A13C5, vertexId=vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564]
2016-05-17 20:45:47,600 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: EF1BAF328A81482DAB50C347146DF1C9 on vertex vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255]
2016-05-17 20:45:47,604 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:47,609 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input EF1BAF328A81482DAB50C347146DF1C9 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:47,609 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:47,609 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:47,607 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=2363FCD16E1448D4978D6CD5181A13C5, vertexName=EE2A5BED0ACD4A2DBB81B997AAE33564, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:47,610 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=2363FCD16E1448D4978D6CD5181A13C5, vertexId=vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564]
2016-05-17 20:45:47,612 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:47,612 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EE2A5BED0ACD4A2DBB81B997AAE33564, vertexId=vertex_1463517947378_0001_1_01, initRequestedTime=1463517947588, initedTime=1463517947611, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 20:45:47,613 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564] transitioned from NEW to INITED due to event V_INIT
2016-05-17 20:45:47,615 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-05-17 20:45:47,619 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: EF1BAF328A81482DAB50C347146DF1C9 on vertex vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255]
2016-05-17 20:45:47,619 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:47,620 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {EF1BAF328A81482DAB50C347146DF1C9=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:47,621 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255] parallelism set to 1
2016-05-17 20:45:47,621 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:47,622 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517947378_0001_1_00, eventCount=1
2016-05-17 20:45:47,629 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:47,630 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E07BAAFDDE004DF18A0716FB41FA3255, vertexId=vertex_1463517947378_0001_1_00, initRequestedTime=1463517947586, initedTime=1463517947628, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:47,630 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255] with distanceFromRoot: 0
2016-05-17 20:45:47,630 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:47,631 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E07BAAFDDE004DF18A0716FB41FA3255
2016-05-17 20:45:47,632 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255]
2016-05-17 20:45:47,636 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:47,636 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] submitted tez dag to app master: application_1463517947378_0001, with dag id: dag_1463517947378_0001_1
2016-05-17 20:45:47,637 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517947378_0001_1_00, startRequestedTime=1463517947631, startedTime=1463517947631
2016-05-17 20:45:47,638 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:47,640 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:47,640 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:TASK_STARTED]: vertexName=E07BAAFDDE004DF18A0716FB41FA3255, taskId=task_1463517947378_0001_1_00_000000, scheduledTime=1463517947639, launchTime=1463517947639
2016-05-17 20:45:47,641 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517947378_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:47,641 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463517947378_0001_1_00 for vertex: vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564] numStartedSources: 1 numSources: 1
2016-05-17 20:45:47,641 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564] with distanceFromRoot: 1
2016-05-17 20:45:47,643 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255]
2016-05-17 20:45:47,644 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E07BAAFDDE004DF18A0716FB41FA3255 in vertex: EE2A5BED0ACD4A2DBB81B997AAE33564
2016-05-17 20:45:47,644 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: EE2A5BED0ACD4A2DBB81B997AAE33564 with 1 source tasks and 1 pending tasks
2016-05-17 20:45:47,645 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:47,645 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517947378_0001_1_01, startRequestedTime=1463517947644, startedTime=1463517947644
2016-05-17 20:45:47,646 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:47,649 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA] count/(1/1) ...splatform/testCount/count, VertexName: E07BAAFDDE004DF18A0716FB41FA3255, VertexParallelism: 1, TaskAttemptID:attempt_1463517947378_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EF1BAF328A81482DAB50C347146DF1C9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EE2A5BED0ACD4A2DBB81B997AAE33564, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:47,651 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517947378_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:47,652 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:47,655 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:47,656 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:47,657 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:47,658 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517947378_0001_00_000001
2016-05-17 20:45:47,659 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517947378_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:47,664 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517947378_0001_00_000001
2016-05-17 20:45:47,664 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:47,665 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:47,668 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517947378_0001_00_000001, launchTime=1463517947665
2016-05-17 20:45:47,668 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517947378_0001_00_000001
2016-05-17 20:45:47,669 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517947378_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:47,670 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517947378_0001_1_00_000000_0] to container: [container_1463517947378_0001_00_000001]
2016-05-17 20:45:47,671 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517947378_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:47,672 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517947378_0001_00_000001 given task: attempt_1463517947378_0001_1_00_000000_0
2016-05-17 20:45:47,672 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517947378_0001_1_00_000000_0] started. Is using containerId: [container_1463517947378_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:47,674 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E07BAAFDDE004DF18A0716FB41FA3255, taskAttemptId=attempt_1463517947378_0001_1_00_000000_0, startTime=1463517947672, containerId=container_1463517947378_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517947378_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:47,674 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:47,676 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517947378_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:47,676 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517947378_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:47,677 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517947378_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517947378_0001_1_00_000000_0
2016-05-17 20:45:47 Starting to run new task attempt: attempt_1463517947378_0001_1_00_000000_0
2016-05-17 20:45:47,678 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517947378_0001_1_00_000000_0
2016-05-17 20:45:47,678 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:47,679 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:47,679 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:47,679 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:47,679 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:47,679 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:47,679 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA] count/(1/1) ...splatform/testCount/count, VertexName: E07BAAFDDE004DF18A0716FB41FA3255, VertexParallelism: 1, TaskAttemptID:attempt_1463517947378_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EF1BAF328A81482DAB50C347146DF1C9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EE2A5BED0ACD4A2DBB81B997AAE33564, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:47,680 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:47,683 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517947378_0001_1_00_000000_0
2016-05-17 20:45:47,686 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:47,687 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:47,689 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517947378_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:47,690 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=EF1BAF328A81482DAB50C347146DF1C9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:47,691 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:47,692 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: EF1BAF328A81482DAB50C347146DF1C9
2016-05-17 20:45:47,692 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: EF1BAF328A81482DAB50C347146DF1C9
2016-05-17 20:45:47,696 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:47,696 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=EE2A5BED0ACD4A2DBB81B997AAE33564, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 20:45:47,696 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:47,697 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: EE2A5BED0ACD4A2DBB81B997AAE33564
2016-05-17 20:45:47,699 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 20:45:47,699 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: EE2A5BED0ACD4A2DBB81B997AAE33564
2016-05-17 20:45:47,701 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: EE2A5BED0ACD4A2DBB81B997AAE33564
2016-05-17 20:45:47,702 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:47,705 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:47,706 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: EF1BAF328A81482DAB50C347146DF1C9
2016-05-17 20:45:47,706 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: EF1BAF328A81482DAB50C347146DF1C9
2016-05-17 20:45:47,716 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E07BAAFDDE004DF18A0716FB41FA3255, ordinal: 0
2016-05-17 20:45:47,716 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E07BAAFDDE004DF18A0716FB41FA3255, mem on start (mb), free: 214, total: 461, max: 672
2016-05-17 20:45:47,716 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:47,717 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:47,717 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:47,718 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:47,718 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:47,719 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:47,719 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:47,720 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 20:45:47,720 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:47,721 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 20:45:47,722 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, EF1BAF328A81482DAB50C347146DF1C9, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:47,722 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, EE2A5BED0ACD4A2DBB81B997AAE33564, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 20:45:47,723 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:47,723 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: EF1BAF328A81482DAB50C347146DF1C9
2016-05-17 20:45:47,723 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:47,723 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: EF1BAF328A81482DAB50C347146DF1C9
2016-05-17 20:45:47,725 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: EF1BAF328A81482DAB50C347146DF1C9 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:47,726 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:47,729 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:47,729 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:47,730 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517947378_0001_1_00_000000_0
2016-05-17 20:45:47,735 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:47,737 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 20:45:47,737 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@21805932
2016-05-17 20:45:47,738 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: EF1BAF328A81482DAB50C347146DF1C9
2016-05-17 20:45:47,739 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: 9019254DB36E43CC8F5BF866381A09A3
2016-05-17 20:45:47,739 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] 9019254DB36E43CC8F5BF866381A09A3
2016-05-17 20:45:47,743 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@21805932
2016-05-17 20:45:47,743 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:47,743 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:47,745 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@622f680b; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@1657dc89; comparator=cascading.tuple.hadoop.util.TupleComparator@624fb1d4; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 20:45:47,746 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 20:45:47,746 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 20:45:47,746 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:47,787 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 20:45:47,787 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 20:45:47,787 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 20:45:47,788 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 20:45:47,788 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 20:45:47,789 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 20:45:47,789 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:47,790 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E07BAAFDDE004DF18A0716FB41FA3255, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:47,793 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517947378_0001_1_00_000000_0
2016-05-17 20:45:47,793 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E07BAAFDDE004DF18A0716FB41FA3255, mem on close (mb), free: 111, total: 461, max: 672
2016-05-17 20:45:47,793 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 20:45:47,793 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 20:45:47,794 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2016-05-17 20:45:47,794 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 20:45:47,801 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 20:45:47 Completed running task attempt: attempt_1463517947378_0001_1_00_000000_0
2016-05-17 20:45:47,805 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517947378_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:47,806 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463517947790, Process_Duration=3, Process_End_Time=1463517947793, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 20:45:47,806 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517947378_0001_1_00, eventCount=1
2016-05-17 20:45:47,807 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:47,806 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:47,808 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:47,808 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E07BAAFDDE004DF18A0716FB41FA3255, taskAttemptId=attempt_1463517947378_0001_1_00_000000_0, startTime=1463517947672, finishTime=1463517947807, timeTaken=135, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517947790, Process_Duration=3, Process_End_Time=1463517947793, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 20:45:47,809 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517947378_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:47,809 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:47,810 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517947378_0001_00_000001
2016-05-17 20:45:47,811 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517947378_0001_1_00_000000_0
2016-05-17 20:45:47,811 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:47,811 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:TASK_FINISHED]: vertexName=E07BAAFDDE004DF18A0716FB41FA3255, taskId=task_1463517947378_0001_1_00_000000, startTime=1463517947672, finishTime=1463517947811, timeTaken=139, status=SUCCEEDED, successfulAttemptID=attempt_1463517947378_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517947790, Process_Duration=3, Process_End_Time=1463517947793, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 20:45:47,813 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517947378_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:47,813 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255] : 1
2016-05-17 20:45:47,813 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255]
2016-05-17 20:45:47,815 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:47,817 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:VERTEX_FINISHED]: vertexName=E07BAAFDDE004DF18A0716FB41FA3255, vertexId=vertex_1463517947378_0001_1_00, initRequestedTime=1463517947586, initedTime=1463517947628, startRequestedTime=1463517947631, startedTime=1463517947631, finishTime=1463517947813, timeTaken=182, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517947790, Process_Duration=3, Process_End_Time=1463517947793, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463517947672, firstTasksToStart=[ task_1463517947378_0001_1_00_000000 ], lastTaskFinishTime=1463517947807, lastTasksToFinish=[ task_1463517947378_0001_1_00_000000 ], minTaskDuration=135, maxTaskDuration=135, avgTaskDuration=135.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517947378_0001_1_00_000000 ], longestDurationTasks=[ task_1463517947378_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:47,818 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:47,818 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517947378_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:47,819 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:47,819 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517947378_0001_00_000001, stoppedTime=1463517947819, exitStatus=0
2016-05-17 20:45:47,821 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517947378_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:47,821 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564] attempt: attempt_1463517947378_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 20:45:47,822 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: EE2A5BED0ACD4A2DBB81B997AAE33564
2016-05-17 20:45:47,822 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564]
2016-05-17 20:45:47,823 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:47,823 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463517947378_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 20:45:47,825 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:47,825 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517947378_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:47,825 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517947378_0001_00_000001: 15 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:47,825 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:TASK_STARTED]: vertexName=EE2A5BED0ACD4A2DBB81B997AAE33564, taskId=task_1463517947378_0001_1_01_000000, scheduledTime=1463517947825, launchTime=1463517947825
2016-05-17 20:45:47,826 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517947378_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:47,826 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517947378_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 20:45:47,826 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517947378_0001_00_000001, Exiting
2016-05-17 20:45:47,826 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517947378_0001_00_000001
2016-05-17 20:45:47,826 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517947378_0001_00_000001
2016-05-17 20:45:47,830 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517947378_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:47,832 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517947378_0001_00_000001 completed successfully
2016-05-17 20:45:47,830 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564]
2016-05-17 20:45:47,833 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517947378_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 20:45:47,833 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA] count/(1/1) ...splatform/testCount/count, VertexName: EE2A5BED0ACD4A2DBB81B997AAE33564, VertexParallelism: 1, TaskAttemptID:attempt_1463517947378_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E07BAAFDDE004DF18A0716FB41FA3255, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=2363FCD16E1448D4978D6CD5181A13C5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:47,833 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517947378_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:47,834 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:47,835 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517947378_0001_00_000002
2016-05-17 20:45:47,836 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517947378_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:47,840 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517947378_0001_00_000002
2016-05-17 20:45:47,841 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 20:45:47,846 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517947378_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:47,846 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:47,847 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517947378_0001_00_000002, launchTime=1463517947845
2016-05-17 20:45:47,848 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517947378_0001_00_000002
2016-05-17 20:45:47,849 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517947378_0001_1_01_000000_0] to container: [container_1463517947378_0001_00_000002]
2016-05-17 20:45:47,850 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517947378_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:47,850 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517947378_0001_00_000002 given task: attempt_1463517947378_0001_1_01_000000_0
2016-05-17 20:45:47,850 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517947378_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517947378_0001_1_01_000000_0
2016-05-17 20:45:47,851 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517947378_0001_1_01_000000_0] started. Is using containerId: [container_1463517947378_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 20:45:47 Starting to run new task attempt: attempt_1463517947378_0001_1_01_000000_0
2016-05-17 20:45:47,851 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517947378_0001_1_01_000000_0
2016-05-17 20:45:47,851 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EE2A5BED0ACD4A2DBB81B997AAE33564, taskAttemptId=attempt_1463517947378_0001_1_01_000000_0, startTime=1463517947850, containerId=container_1463517947378_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517947378_0001_00_000002/teamcity, completedLogs=
2016-05-17 20:45:47,851 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:47,852 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:47,852 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:47,852 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:47,852 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517947378_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:47,852 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:47,852 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517947378_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:47,852 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:47,853 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:47,853 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA] count/(1/1) ...splatform/testCount/count, VertexName: EE2A5BED0ACD4A2DBB81B997AAE33564, VertexParallelism: 1, TaskAttemptID:attempt_1463517947378_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E07BAAFDDE004DF18A0716FB41FA3255, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=2363FCD16E1448D4978D6CD5181A13C5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:47,853 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:47,854 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517947378_0001_1_01_000000_0
2016-05-17 20:45:47,855 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:47,855 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:47,856 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517947378_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 20:45:47,857 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E07BAAFDDE004DF18A0716FB41FA3255, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 20:45:47,857 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=2363FCD16E1448D4978D6CD5181A13C5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:47,857 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:47,859 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:47,859 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E07BAAFDDE004DF18A0716FB41FA3255
2016-05-17 20:45:47,860 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:47,860 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 2363FCD16E1448D4978D6CD5181A13C5
2016-05-17 20:45:47,861 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:47,861 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 2363FCD16E1448D4978D6CD5181A13C5
2016-05-17 20:45:47,865 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:47,869 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 20:45:47,869 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 20:45:47,869 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: E07BAAFDDE004DF18A0716FB41FA3255
2016-05-17 20:45:47,870 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E07BAAFDDE004DF18A0716FB41FA3255
2016-05-17 20:45:47,882 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EE2A5BED0ACD4A2DBB81B997AAE33564, ordinal: 1
2016-05-17 20:45:47,882 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EE2A5BED0ACD4A2DBB81B997AAE33564, mem on start (mb), free: 102, total: 461, max: 672
2016-05-17 20:45:47,882 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:47,882 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:47,883 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:47,890 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:47,890 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 2363FCD16E1448D4978D6CD5181A13C5
2016-05-17 20:45:47,891 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:47,891 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:47,891 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:47,891 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:47,891 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 20:45:47,891 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:47,892 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 20:45:47,892 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 2363FCD16E1448D4978D6CD5181A13C5, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:47,892 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E07BAAFDDE004DF18A0716FB41FA3255, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 20:45:47,892 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:47,892 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E07BAAFDDE004DF18A0716FB41FA3255 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:47,892 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E07BAAFDDE004DF18A0716FB41FA3255
2016-05-17 20:45:47,892 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:47,896 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:47,896 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:47,897 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 20:45:47,897 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E07BAAFDDE004DF18A0716FB41FA3255 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 20:45:47,897 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 20:45:47,897 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 20:45:47,898 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 20:45:47,900 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E07BAAFDDE004DF18A0716FB41FA3255
2016-05-17 20:45:47,900 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:47,900 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517947378_0001_1_01_000000_0
2016-05-17 20:45:47,901 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463517947378_0001_1_00_000000_0_10012, runDuration: 106546, ]
2016-05-17 20:45:47,903 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463517947378_0001_1_00_000000_0_10012] to DISK_DIRECT, CompressedSize=463, DecompressedSize=459,EndTime=1463517947903, TimeTaken=2, Rate=0.22 MB/s
2016-05-17 20:45:47,903 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E07BAAFDDE004DF18A0716FB41FA3255
2016-05-17 20:45:47,904 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 20:45:47,904 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [E07BAAFDDE004DF18A0716FB41FA3255] #3 in 3ms
2016-05-17 20:45:47,905 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E07BAAFDDE004DF18A0716FB41FA3255] #3, status:false, isInterrupted:false
2016-05-17 20:45:47,907 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 20:45:47,908 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 463 bytes from disk
2016-05-17 20:45:47,908 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:47,908 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 20:45:47,909 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-05-17 20:45:47,909 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E07BAAFDDE004DF18A0716FB41FA3255
2016-05-17 20:45:47,911 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 20:45:47,912 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-05-17 20:45:47,914 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: 9019254DB36E43CC8F5BF866381A09A3
2016-05-17 20:45:47,914 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count"], id: 2363FCD16E1448D4978D6CD5181A13C5
2016-05-17 20:45:47,915 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count"]
2016-05-17 20:45:47,915 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] 9019254DB36E43CC8F5BF866381A09A3, for 1 inputs
2016-05-17 20:45:47,916 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EE2A5BED0ACD4A2DBB81B997AAE33564, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:47,918 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@5eb3589d; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@507edbfd
2016-05-17 20:45:47,923 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:47,924 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:47,927 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517947378_0001_1_01_000000_0
2016-05-17 20:45:47,928 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517947378_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 20:45:47,929 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517947378_0001_1_01_000000_0
2016-05-17 20:45:47,929 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EE2A5BED0ACD4A2DBB81B997AAE33564, mem on close (mb), free: 97, total: 461, max: 672
2016-05-17 20:45:47,929 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E07BAAFDDE004DF18A0716FB41FA3255
2016-05-17 20:45:47,929 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 20:45:47,930 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 20:45:47,930 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:47,930 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 20:45:47,930 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 20:45:47 Completed running task attempt: attempt_1463517947378_0001_1_01_000000_0
2016-05-17 20:45:47,931 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517947378_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:47,935 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:47,937 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EE2A5BED0ACD4A2DBB81B997AAE33564, taskAttemptId=attempt_1463517947378_0001_1_01_000000_0, startTime=1463517947850, finishTime=1463517947934, timeTaken=84, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=802, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517947916, Process_Duration=13, Process_End_Time=1463517947929, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 20:45:47,937 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517947378_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:47,937 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:47,937 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517947378_0001_1_01_000000_0
2016-05-17 20:45:47,938 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:47,939 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:TASK_FINISHED]: vertexName=EE2A5BED0ACD4A2DBB81B997AAE33564, taskId=task_1463517947378_0001_1_01_000000, startTime=1463517947850, finishTime=1463517947938, timeTaken=88, status=SUCCEEDED, successfulAttemptID=attempt_1463517947378_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=802, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517947916, Process_Duration=13, Process_End_Time=1463517947929, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 20:45:47,939 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517947378_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:47,940 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564] : 1
2016-05-17 20:45:47,940 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564]
2016-05-17 20:45:47,941 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=802, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463517947916, Process_Duration=13, Process_End_Time=1463517947929, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=17]]
2016-05-17 20:45:47,942 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:47,942 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:47,943 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517947378_0001_00_000002
2016-05-17 20:45:47,943 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:47,945 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:VERTEX_FINISHED]: vertexName=EE2A5BED0ACD4A2DBB81B997AAE33564, vertexId=vertex_1463517947378_0001_1_01, initRequestedTime=1463517947588, initedTime=1463517947611, startRequestedTime=1463517947644, startedTime=1463517947644, finishTime=1463517947940, timeTaken=296, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=802, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517947916, Process_Duration=13, Process_End_Time=1463517947929, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1463517947850, firstTasksToStart=[ task_1463517947378_0001_1_01_000000 ], lastTaskFinishTime=1463517947934, lastTasksToFinish=[ task_1463517947378_0001_1_01_000000 ], minTaskDuration=84, maxTaskDuration=84, avgTaskDuration=84.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517947378_0001_1_01_000000 ], longestDurationTasks=[ task_1463517947378_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:47,945 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:47,945 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517947378_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:47,946 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517947378_0001_00_000002, stoppedTime=1463517947946, exitStatus=0
2016-05-17 20:45:47,946 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517947378_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:47,946 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:47,947 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:47,947 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517947378_0001_1
2016-05-17 20:45:47,948 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517947378_0001_1
2016-05-17 20:45:47,948 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463517947378_0001_1_00 [E07BAAFDDE004DF18A0716FB41FA3255]
2016-05-17 20:45:47,948 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 2363FCD16E1448D4978D6CD5181A13C5 for vertex: vertex_1463517947378_0001_1_01 [EE2A5BED0ACD4A2DBB81B997AAE33564]
2016-05-17 20:45:47,953 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517947378_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 20:45:47,954 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517947378_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:47,954 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517947378_0001_00_000002, Exiting
2016-05-17 20:45:47,954 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517947378_0001_00_000002
2016-05-17 20:45:47,954 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517947378_0001_00_000002
2016-05-17 20:45:47,955 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517947378_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:47,955 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517947378_0001_00_000002 completed successfully
2016-05-17 20:45:47,964 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517947378_0001_1, queueSize=0
2016-05-17 20:45:47,964 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:47,966 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517947378_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517947378_0001_1, startTime=1463517947585, finishTime=1463517947960, timeTaken=375, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=940, FILE_BYTES_WRITTEN=1297, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=966787072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927035895706, Process_Duration=16, Process_End_Time=2927035895722, Read_Duration=4, Tuples_Read=40, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-05-17 20:45:47,966 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517947378_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:47,966 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517947378_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:47,966 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517947378_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:47,967 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517947378_0001_00_000002
2016-05-17 20:45:47 Completed Dag: dag_1463517947378_0001_1
2016-05-17 20:45:47,967 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517947378_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:47,968 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517947378_0001_1_post
2016-05-17 20:45:47,968 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:47,968 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:47,978 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...splatform/testCount/count, applicationId=application_1463517947378_0001
2016-05-17 20:45:47,981 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:47,982 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:47,982 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:47,982 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:47,982 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:47,983 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:47,983 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:47,983 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:47,984 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:47,984 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:47,983 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:47,984 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:47,986 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:47,991 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/79BB887D3A22413B82E4676B966FBA17/AA257C0A22C54A3CA183723B05BBCCEA/.tez/application_1463517947378_0001
2016-05-17 20:45:47,991 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count/_temporary
2016-05-17 20:45:47,992 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] completed in: 00:00.643
2016-05-17 20:45:48,089 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount/count
2016-05-17 20:45:48,129 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:48,129 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:48,193 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.022
2016-05-17 20:45:48,208 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.037
2016-05-17 20:45:48,209 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 20:45:48,209 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 20:45:48,210 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:48,232 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 20:45:48,233 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap
2016-05-17 20:45:48,245 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 20:45:48,245 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:48,246 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap"]
2016-05-17 20:45:48,246 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 20:45:48,246 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 20:45:48,246 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 20:45:48,247 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...pesplatform/testSwap/swap
2016-05-17 20:45:48,247 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:48,267 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 20:45:48,272 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:48,272 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:48,273 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:48,282 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/EED6E71F2DEE47E0A3353AF65DDD9B4B/A9F7BDF0E3BD4B1E99AEE6170A20A1FD/.tez/application_1463517948272_0001 doesn't exist and is created
2016-05-17 20:45:48,303 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:48,303 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:48,305 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/EED6E71F2DEE47E0A3353AF65DDD9B4B/A9F7BDF0E3BD4B1E99AEE6170A20A1FD/.tez/application_1463517948272_0001
2016-05-17 20:45:48,310 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517948272_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:48,326 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:48,327 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:48,329 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:48,329 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:48,330 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/EED6E71F2DEE47E0A3353AF65DDD9B4B/A9F7BDF0E3BD4B1E99AEE6170A20A1FD/.tez/application_1463517948272_0001/localmode-log-dir
2016-05-17 20:45:48,330 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/EED6E71F2DEE47E0A3353AF65DDD9B4B/A9F7BDF0E3BD4B1E99AEE6170A20A1FD/.tez/application_1463517948272_0001/localmode-log-dir/history.txt.appattempt_1463517948272_0001_000000, maxErrors=10
2016-05-17 20:45:48,330 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:48,331 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517948272_0001_000000, appSubmitTime=1463517948309, launchTime=1463517948309
2016-05-17 20:45:48,361 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:48,369 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:44085
2016-05-17 20:45:48,371 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:48,373 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:48,373 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:48,374 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:48,377 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517948272_0001_000000, startTime=1463517948377
2016-05-17 20:45:48,377 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:48,377 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:48,404 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:48,404 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:48,404 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...pesplatform/testSwap/swap, applicationId=application_1463517948272_0001, dagName=[EED6E71F2DEE47E0A3353AF65DDD9B4B/A9F7BDF0E3BD4B1E99AEE6170A20A1FD] test/(1/1) ...pesplatform/testSwap/swap
2016-05-17 20:45:48,425 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [EED6E71F2DEE47E0A3353AF65DDD9B4B/A9F7BDF0E3BD4B1E99AEE6170A20A1FD] test/(1/1) ...pesplatform/testSwap/swap
2016-05-17 20:45:48,427 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517948272_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/EED6E71F2DEE47E0A3353AF65DDD9B4B/A9F7BDF0E3BD4B1E99AEE6170A20A1FD/.tez/application_1463517948272_0001/localmode-log-dir/dag_1463517948272_0001_1.dot
2016-05-17 20:45:48,428 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/EED6E71F2DEE47E0A3353AF65DDD9B4B/A9F7BDF0E3BD4B1E99AEE6170A20A1FD/.tez/application_1463517948272_0001/localmode-log-dir/dag_1463517948272_0001_1-tez-dag.pb.txt
2016-05-17 20:45:48,451 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517948272_0001_1
2016-05-17 20:45:48,451 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:48,451 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [EED6E71F2DEE47E0A3353AF65DDD9B4B/A9F7BDF0E3BD4B1E99AEE6170A20A1FD] test/(1/1) ...pesplatform/testSwap/swap
2016-05-17 20:45:48 Running Dag: dag_1463517948272_0001_1
2016-05-17 20:45:48,464 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517948272_0001_1, submitTime=1463517948425
2016-05-17 20:45:48,464 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D9053C7DA5184C128BE4DC7D95419D4D
2016-05-17 20:45:48,465 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:48,465 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:48,466 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:48,466 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517948272_0001_1, initTime=1463517948464
2016-05-17 20:45:48,466 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517948272_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:48,467 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:DAG_STARTED]: dagID=dag_1463517948272_0001_1, startTime=1463517948467
2016-05-17 20:45:48,467 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:48,467 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517948272_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:48,468 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D9053C7DA5184C128BE4DC7D95419D4D : {87C55A64C93A44A2ABBD366B2E9FA4DC={InputName=87C55A64C93A44A2ABBD366B2E9FA4DC}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:48,468 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 87C55A64C93A44A2ABBD366B2E9FA4DC, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:48,468 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D]
2016-05-17 20:45:48,468 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D]
2016-05-17 20:45:48,470 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D]
2016-05-17 20:45:48,467 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:48,470 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D]. Starting root input initializers: 1
2016-05-17 20:45:48,471 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:48,472 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...pesplatform/testSwap/swap, applicationId=application_1463517948272_0001, dagName=[EED6E71F2DEE47E0A3353AF65DDD9B4B/A9F7BDF0E3BD4B1E99AEE6170A20A1FD] test/(1/1) ...pesplatform/testSwap/swap
2016-05-17 20:45:48,472 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 87C55A64C93A44A2ABBD366B2E9FA4DC on vertex vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D]
2016-05-17 20:45:48,474 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:48,474 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 87C55A64C93A44A2ABBD366B2E9FA4DC asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:48,475 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:48,475 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:48,478 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 20:45:48,478 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 87C55A64C93A44A2ABBD366B2E9FA4DC on vertex vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D]
2016-05-17 20:45:48,479 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:48,479 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {87C55A64C93A44A2ABBD366B2E9FA4DC=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:48,479 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D] parallelism set to 1
2016-05-17 20:45:48,479 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:48,480 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517948272_0001_1_00, eventCount=1
2016-05-17 20:45:48,480 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D]
2016-05-17 20:45:48,481 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=F41FA8993CFD46A180CF5ADC68F147B2, vertexId=vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:48,481 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=F41FA8993CFD46A180CF5ADC68F147B2, vertexId=vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D]
2016-05-17 20:45:48,483 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=F41FA8993CFD46A180CF5ADC68F147B2, vertexName=D9053C7DA5184C128BE4DC7D95419D4D, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:48,483 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=F41FA8993CFD46A180CF5ADC68F147B2, vertexId=vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D]
2016-05-17 20:45:48,485 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:48,485 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D9053C7DA5184C128BE4DC7D95419D4D, vertexId=vertex_1463517948272_0001_1_00, initRequestedTime=1463517948468, initedTime=1463517948485, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:48,485 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D] with distanceFromRoot: 0
2016-05-17 20:45:48,486 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:48,486 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D9053C7DA5184C128BE4DC7D95419D4D
2016-05-17 20:45:48,486 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D]
2016-05-17 20:45:48,486 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:48,486 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517948272_0001_1_00, startRequestedTime=1463517948486, startedTime=1463517948486
2016-05-17 20:45:48,487 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:48,488 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:48,488 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:TASK_STARTED]: vertexName=D9053C7DA5184C128BE4DC7D95419D4D, taskId=task_1463517948272_0001_1_00_000000, scheduledTime=1463517948487, launchTime=1463517948487
2016-05-17 20:45:48,488 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517948272_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:48,488 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D]
2016-05-17 20:45:48,489 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [EED6E71F2DEE47E0A3353AF65DDD9B4B/A9F7BDF0E3BD4B1E99AEE6170A20A1FD] test/(1/1) ...pesplatform/testSwap/swap, VertexName: D9053C7DA5184C128BE4DC7D95419D4D, VertexParallelism: 1, TaskAttemptID:attempt_1463517948272_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=87C55A64C93A44A2ABBD366B2E9FA4DC, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F41FA8993CFD46A180CF5ADC68F147B2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:48,489 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517948272_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:48,490 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:48,491 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:48,492 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:48,497 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:48,497 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517948272_0001_00_000001
2016-05-17 20:45:48,497 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517948272_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:48,500 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517948272_0001_00_000001
2016-05-17 20:45:48,500 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:48,503 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517948272_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:48,503 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:48,503 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463517948272_0001, with dag id: dag_1463517948272_0001_1
2016-05-17 20:45:48,504 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517948272_0001_00_000001, launchTime=1463517948502
2016-05-17 20:45:48,506 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517948272_0001_00_000001
2016-05-17 20:45:48,510 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517948272_0001_1_00_000000_0] to container: [container_1463517948272_0001_00_000001]
2016-05-17 20:45:48,510 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517948272_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:48,510 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517948272_0001_00_000001 given task: attempt_1463517948272_0001_1_00_000000_0
2016-05-17 20:45:48,511 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517948272_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517948272_0001_1_00_000000_0
2016-05-17 20:45:48 Starting to run new task attempt: attempt_1463517948272_0001_1_00_000000_0
2016-05-17 20:45:48,512 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517948272_0001_1_00_000000_0
2016-05-17 20:45:48,512 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:48,512 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517948272_0001_1_00_000000_0] started. Is using containerId: [container_1463517948272_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:48,512 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:48,512 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D9053C7DA5184C128BE4DC7D95419D4D, taskAttemptId=attempt_1463517948272_0001_1_00_000000_0, startTime=1463517948512, containerId=container_1463517948272_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517948272_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:48,513 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:48,512 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:48,513 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:48,513 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:48,514 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:48,514 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [EED6E71F2DEE47E0A3353AF65DDD9B4B/A9F7BDF0E3BD4B1E99AEE6170A20A1FD] test/(1/1) ...pesplatform/testSwap/swap, VertexName: D9053C7DA5184C128BE4DC7D95419D4D, VertexParallelism: 1, TaskAttemptID:attempt_1463517948272_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=87C55A64C93A44A2ABBD366B2E9FA4DC, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F41FA8993CFD46A180CF5ADC68F147B2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:48,514 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:48,513 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517948272_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:48,515 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517948272_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:48,516 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517948272_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:48,516 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517948272_0001_1_00_000000_0
2016-05-17 20:45:48,516 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:48,517 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:48,517 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=87C55A64C93A44A2ABBD366B2E9FA4DC, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:48,518 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:48,518 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 87C55A64C93A44A2ABBD366B2E9FA4DC
2016-05-17 20:45:48,518 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 87C55A64C93A44A2ABBD366B2E9FA4DC
2016-05-17 20:45:48,520 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:48,523 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:48,525 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F41FA8993CFD46A180CF5ADC68F147B2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:48,525 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:48,525 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 87C55A64C93A44A2ABBD366B2E9FA4DC
2016-05-17 20:45:48,525 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:48,525 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 87C55A64C93A44A2ABBD366B2E9FA4DC
2016-05-17 20:45:48,534 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F41FA8993CFD46A180CF5ADC68F147B2
2016-05-17 20:45:48,534 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:48,534 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: F41FA8993CFD46A180CF5ADC68F147B2
2016-05-17 20:45:48,541 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D9053C7DA5184C128BE4DC7D95419D4D, ordinal: 0
2016-05-17 20:45:48,541 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D9053C7DA5184C128BE4DC7D95419D4D, mem on start (mb), free: 239, total: 461, max: 672
2016-05-17 20:45:48,542 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:48,542 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:48,542 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:48,546 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:48,546 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F41FA8993CFD46A180CF5ADC68F147B2
2016-05-17 20:45:48,547 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:48,547 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:48,547 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:48,548 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:48,548 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:48,548 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:48,548 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:48,549 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 20:45:48,549 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 87C55A64C93A44A2ABBD366B2E9FA4DC, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:48,549 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F41FA8993CFD46A180CF5ADC68F147B2, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:48,549 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:48,550 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 87C55A64C93A44A2ABBD366B2E9FA4DC
2016-05-17 20:45:48,550 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:48,550 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 87C55A64C93A44A2ABBD366B2E9FA4DC
2016-05-17 20:45:48,550 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 87C55A64C93A44A2ABBD366B2E9FA4DC being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:48,550 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:48,551 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:48,551 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:48,552 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517948272_0001_1_00_000000_0
2016-05-17 20:45:48,552 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:48,553 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@360b662b
2016-05-17 20:45:48,556 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@360b662b
2016-05-17 20:45:48,556 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:48,557 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:48,557 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:48,558 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 87C55A64C93A44A2ABBD366B2E9FA4DC
2016-05-17 20:45:48,559 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap"], id: F41FA8993CFD46A180CF5ADC68F147B2
2016-05-17 20:45:48,559 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap"]
2016-05-17 20:45:48,560 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:48,560 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:48,560 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D9053C7DA5184C128BE4DC7D95419D4D, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:48,563 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:48,563 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:48,564 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517948272_0001_1_00_000000_0
2016-05-17 20:45:48,564 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517948272_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 20:45:48,565 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517948272_0001_1_00_000000_0
2016-05-17 20:45:48,565 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D9053C7DA5184C128BE4DC7D95419D4D, mem on close (mb), free: 237, total: 461, max: 672
2016-05-17 20:45:48 Completed running task attempt: attempt_1463517948272_0001_1_00_000000_0
2016-05-17 20:45:48,567 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517948272_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:48,568 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D9053C7DA5184C128BE4DC7D95419D4D, taskAttemptId=attempt_1463517948272_0001_1_00_000000_0, startTime=1463517948512, finishTime=1463517948568, timeTaken=56, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=712, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463517948561, Process_Duration=4, Process_End_Time=1463517948565, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-17 20:45:48,569 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:48,569 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517948272_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:48,570 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:48,570 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517948272_0001_1_00_000000_0
2016-05-17 20:45:48,571 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:48,571 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:TASK_FINISHED]: vertexName=D9053C7DA5184C128BE4DC7D95419D4D, taskId=task_1463517948272_0001_1_00_000000, startTime=1463517948512, finishTime=1463517948570, timeTaken=58, status=SUCCEEDED, successfulAttemptID=attempt_1463517948272_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=712, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463517948561, Process_Duration=4, Process_End_Time=1463517948565, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-17 20:45:48,571 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517948272_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:48,571 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D] : 1
2016-05-17 20:45:48,571 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D]
2016-05-17 20:45:48,572 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=712, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10][cascading.flow.SliceCounters Process_Begin_Time=1463517948561, Process_Duration=4, Process_End_Time=1463517948565, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10, Tuples_Written=10]]
2016-05-17 20:45:48,573 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:VERTEX_FINISHED]: vertexName=D9053C7DA5184C128BE4DC7D95419D4D, vertexId=vertex_1463517948272_0001_1_00, initRequestedTime=1463517948468, initedTime=1463517948485, startRequestedTime=1463517948486, startedTime=1463517948486, finishTime=1463517948571, timeTaken=85, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=712, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463517948561, Process_Duration=4, Process_End_Time=1463517948565, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10, vertexStats=firstTaskStartTime=1463517948512, firstTasksToStart=[ task_1463517948272_0001_1_00_000000 ], lastTaskFinishTime=1463517948568, lastTasksToFinish=[ task_1463517948272_0001_1_00_000000 ], minTaskDuration=56, maxTaskDuration=56, avgTaskDuration=56.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517948272_0001_1_00_000000 ], longestDurationTasks=[ task_1463517948272_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:48,574 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:48,574 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517948272_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:48,573 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:48,576 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517948272_0001_00_000001, stoppedTime=1463517948576, exitStatus=0
2016-05-17 20:45:48,576 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517948272_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:48,577 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 20:45:48,577 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517948272_0001_1
2016-05-17 20:45:48,578 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517948272_0001_1
2016-05-17 20:45:48,578 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: F41FA8993CFD46A180CF5ADC68F147B2 for vertex: vertex_1463517948272_0001_1_00 [D9053C7DA5184C128BE4DC7D95419D4D]
2016-05-17 20:45:48,579 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:48,579 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:48,576 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:48,580 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517948272_0001_00_000001
2016-05-17 20:45:48,582 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517948272_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:48,582 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517948272_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:48,583 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517948272_0001_00_000001, Exiting
2016-05-17 20:45:48,583 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517948272_0001_00_000001
2016-05-17 20:45:48,583 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517948272_0001_00_000001
2016-05-17 20:45:48,585 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517948272_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:48,586 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517948272_0001_00_000001 completed successfully
2016-05-17 20:45:48,591 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517948272_0001_1, queueSize=0
2016-05-17 20:45:48,592 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948272_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517948272_0001_1, startTime=1463517948467, finishTime=1463517948590, timeTaken=123, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=712, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463517948561, Process_Duration=4, Process_End_Time=1463517948565, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-17 20:45:48,592 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517948272_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:48,592 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517948272_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:48,592 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517948272_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:48,592 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517948272_0001_00_000001
2016-05-17 20:45:48 Completed Dag: dag_1463517948272_0001_1
2016-05-17 20:45:48,592 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517948272_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:48,593 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517948272_0001_1_post
2016-05-17 20:45:48,593 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:48,593 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:48,593 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:48,601 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...pesplatform/testSwap/swap, applicationId=application_1463517948272_0001
2016-05-17 20:45:48,603 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:48,603 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:48,603 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:48,603 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:48,603 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:48,604 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:48,604 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:48,605 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:48,605 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:48,606 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:48,606 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:48,607 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:48,607 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:48,615 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/EED6E71F2DEE47E0A3353AF65DDD9B4B/A9F7BDF0E3BD4B1E99AEE6170A20A1FD/.tez/application_1463517948272_0001
2016-05-17 20:45:48,616 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap/_temporary
2016-05-17 20:45:48,617 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.371
2016-05-17 20:45:48,656 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSwap/swap
2016-05-17 20:45:48,685 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:48,685 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:48,729 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-17 20:45:48,741 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.027
2016-05-17 20:45:48,742 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 20:45:48,742 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 20:45:48,743 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:48,760 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower+upper.txt
2016-05-17 20:45:48,761 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped
2016-05-17 20:45:48,776 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 20:45:48,777 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower+upper.txt"]
2016-05-17 20:45:48,777 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped"]
2016-05-17 20:45:48,777 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 20:45:48,777 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 20:45:48,778 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 20:45:48,779 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...orm/testUnGroup/ungrouped
2016-05-17 20:45:48,779 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:48,795 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 20:45:48,800 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:48,800 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:48,800 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:48,811 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/47573793A067401B91C8E169DCD8A63D/DAB5F219900741AC80176FC89FD7907E/.tez/application_1463517948800_0001 doesn't exist and is created
2016-05-17 20:45:48,834 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:48,834 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:48,835 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/47573793A067401B91C8E169DCD8A63D/DAB5F219900741AC80176FC89FD7907E/.tez/application_1463517948800_0001
2016-05-17 20:45:48,840 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517948800_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:48,852 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:48,853 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:48,855 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:48,855 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:48,855 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/47573793A067401B91C8E169DCD8A63D/DAB5F219900741AC80176FC89FD7907E/.tez/application_1463517948800_0001/localmode-log-dir
2016-05-17 20:45:48,855 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/47573793A067401B91C8E169DCD8A63D/DAB5F219900741AC80176FC89FD7907E/.tez/application_1463517948800_0001/localmode-log-dir/history.txt.appattempt_1463517948800_0001_000000, maxErrors=10
2016-05-17 20:45:48,856 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:48,856 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517948800_0001_000000, appSubmitTime=1463517948839, launchTime=1463517948839
2016-05-17 20:45:48,861 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:48,863 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:48,865 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:48,867 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:48,869 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:38845
2016-05-17 20:45:48,871 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:48,874 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517948800_0001_000000, startTime=1463517948874
2016-05-17 20:45:48,874 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:48,874 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:48,934 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:48,934 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:48,935 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...orm/testUnGroup/ungrouped, applicationId=application_1463517948800_0001, dagName=[47573793A067401B91C8E169DCD8A63D/DAB5F219900741AC80176FC89FD7907E] test/(1/1) ...orm/testUnGroup/ungrouped
2016-05-17 20:45:48,957 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [47573793A067401B91C8E169DCD8A63D/DAB5F219900741AC80176FC89FD7907E] test/(1/1) ...orm/testUnGroup/ungrouped
2016-05-17 20:45:48,960 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517948800_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/47573793A067401B91C8E169DCD8A63D/DAB5F219900741AC80176FC89FD7907E/.tez/application_1463517948800_0001/localmode-log-dir/dag_1463517948800_0001_1.dot
2016-05-17 20:45:48,961 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/47573793A067401B91C8E169DCD8A63D/DAB5F219900741AC80176FC89FD7907E/.tez/application_1463517948800_0001/localmode-log-dir/dag_1463517948800_0001_1-tez-dag.pb.txt
2016-05-17 20:45:48,990 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517948800_0001_1
2016-05-17 20:45:48,991 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:48,991 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [47573793A067401B91C8E169DCD8A63D/DAB5F219900741AC80176FC89FD7907E] test/(1/1) ...orm/testUnGroup/ungrouped
2016-05-17 20:45:48 Running Dag: dag_1463517948800_0001_1
2016-05-17 20:45:49,004 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:49,004 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517948800_0001_1, submitTime=1463517948957
2016-05-17 20:45:49,005 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F8590CB2C4D643849371443AE7B4080B
2016-05-17 20:45:49,005 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:49,006 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:49,006 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517948800_0001_1, initTime=1463517949005
2016-05-17 20:45:49,006 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517948800_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:49,007 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:DAG_STARTED]: dagID=dag_1463517948800_0001_1, startTime=1463517949007
2016-05-17 20:45:49,007 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:49,007 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517948800_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:49,007 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F8590CB2C4D643849371443AE7B4080B : {70DA86223CD7470C90C12B19E2874E00={InputName=70DA86223CD7470C90C12B19E2874E00}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:49,007 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:49,007 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 70DA86223CD7470C90C12B19E2874E00, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:49,008 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B]
2016-05-17 20:45:49,008 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B]
2016-05-17 20:45:49,008 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B]
2016-05-17 20:45:49,008 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B]. Starting root input initializers: 1
2016-05-17 20:45:49,009 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:49,010 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 70DA86223CD7470C90C12B19E2874E00 on vertex vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B]
2016-05-17 20:45:49,010 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...orm/testUnGroup/ungrouped, applicationId=application_1463517948800_0001, dagName=[47573793A067401B91C8E169DCD8A63D/DAB5F219900741AC80176FC89FD7907E] test/(1/1) ...orm/testUnGroup/ungrouped
2016-05-17 20:45:49,014 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:49,014 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 70DA86223CD7470C90C12B19E2874E00 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:49,015 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:49,015 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:49,018 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 412
2016-05-17 20:45:49,021 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 70DA86223CD7470C90C12B19E2874E00 on vertex vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B]
2016-05-17 20:45:49,021 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:49,021 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {70DA86223CD7470C90C12B19E2874E00=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:49,022 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B] parallelism set to 1
2016-05-17 20:45:49,022 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:49,022 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517948800_0001_1_00, eventCount=1
2016-05-17 20:45:49,023 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B]
2016-05-17 20:45:49,024 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=D03628992BF34FA1A5488F67378A92C5, vertexId=vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:49,024 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=D03628992BF34FA1A5488F67378A92C5, vertexId=vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B]
2016-05-17 20:45:49,026 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=D03628992BF34FA1A5488F67378A92C5, vertexName=F8590CB2C4D643849371443AE7B4080B, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:49,026 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=D03628992BF34FA1A5488F67378A92C5, vertexId=vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B]
2016-05-17 20:45:49,028 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F8590CB2C4D643849371443AE7B4080B, vertexId=vertex_1463517948800_0001_1_00, initRequestedTime=1463517949007, initedTime=1463517949028, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:49,028 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B] with distanceFromRoot: 0
2016-05-17 20:45:49,028 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:49,028 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:49,029 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F8590CB2C4D643849371443AE7B4080B
2016-05-17 20:45:49,029 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B]
2016-05-17 20:45:49,029 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:49,029 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517948800_0001_1_00, startRequestedTime=1463517949029, startedTime=1463517949029
2016-05-17 20:45:49,030 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:49,030 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:49,030 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:TASK_STARTED]: vertexName=F8590CB2C4D643849371443AE7B4080B, taskId=task_1463517948800_0001_1_00_000000, scheduledTime=1463517949030, launchTime=1463517949030
2016-05-17 20:45:49,030 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517948800_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:49,031 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B]
2016-05-17 20:45:49,031 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [47573793A067401B91C8E169DCD8A63D/DAB5F219900741AC80176FC89FD7907E] test/(1/1) ...orm/testUnGroup/ungrouped, VertexName: F8590CB2C4D643849371443AE7B4080B, VertexParallelism: 1, TaskAttemptID:attempt_1463517948800_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=70DA86223CD7470C90C12B19E2874E00, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D03628992BF34FA1A5488F67378A92C5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:49,031 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517948800_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:49,031 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:49,035 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463517948800_0001, with dag id: dag_1463517948800_0001_1
2016-05-17 20:45:49,038 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:49,040 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:49,040 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:49,041 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517948800_0001_00_000001
2016-05-17 20:45:49,041 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517948800_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:49,043 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517948800_0001_00_000001
2016-05-17 20:45:49,043 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:49,045 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517948800_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:49,045 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:49,046 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517948800_0001_00_000001, launchTime=1463517949045
2016-05-17 20:45:49,047 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517948800_0001_00_000001
2016-05-17 20:45:49,048 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517948800_0001_1_00_000000_0] to container: [container_1463517948800_0001_00_000001]
2016-05-17 20:45:49,049 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517948800_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:49,049 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517948800_0001_00_000001 given task: attempt_1463517948800_0001_1_00_000000_0
2016-05-17 20:45:49,049 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517948800_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517948800_0001_1_00_000000_0
2016-05-17 20:45:49,049 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517948800_0001_1_00_000000_0] started. Is using containerId: [container_1463517948800_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:49,049 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F8590CB2C4D643849371443AE7B4080B, taskAttemptId=attempt_1463517948800_0001_1_00_000000_0, startTime=1463517949049, containerId=container_1463517948800_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517948800_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:49,050 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517948800_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:49,050 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517948800_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:49,050 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:49 Starting to run new task attempt: attempt_1463517948800_0001_1_00_000000_0
2016-05-17 20:45:49,051 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517948800_0001_1_00_000000_0
2016-05-17 20:45:49,051 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:49,051 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:49,051 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:49,051 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:49,052 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:49,052 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:49,052 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [47573793A067401B91C8E169DCD8A63D/DAB5F219900741AC80176FC89FD7907E] test/(1/1) ...orm/testUnGroup/ungrouped, VertexName: F8590CB2C4D643849371443AE7B4080B, VertexParallelism: 1, TaskAttemptID:attempt_1463517948800_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=70DA86223CD7470C90C12B19E2874E00, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D03628992BF34FA1A5488F67378A92C5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:49,052 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:49,053 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517948800_0001_1_00_000000_0
2016-05-17 20:45:49,056 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:49,056 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:49,056 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517948800_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:49,058 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=70DA86223CD7470C90C12B19E2874E00, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:49,058 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:49,058 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D03628992BF34FA1A5488F67378A92C5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:49,060 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:49,061 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:49,061 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:49,061 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 70DA86223CD7470C90C12B19E2874E00
2016-05-17 20:45:49,061 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D03628992BF34FA1A5488F67378A92C5
2016-05-17 20:45:49,061 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:49,062 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: D03628992BF34FA1A5488F67378A92C5
2016-05-17 20:45:49,061 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 70DA86223CD7470C90C12B19E2874E00
2016-05-17 20:45:49,066 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:49,066 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 70DA86223CD7470C90C12B19E2874E00
2016-05-17 20:45:49,067 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 70DA86223CD7470C90C12B19E2874E00
2016-05-17 20:45:49,074 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F8590CB2C4D643849371443AE7B4080B, ordinal: 0
2016-05-17 20:45:49,074 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F8590CB2C4D643849371443AE7B4080B, mem on start (mb), free: 159, total: 461, max: 672
2016-05-17 20:45:49,074 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:49,074 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:49,074 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:49,076 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:49,077 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D03628992BF34FA1A5488F67378A92C5
2016-05-17 20:45:49,077 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:49,077 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:49,077 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:49,077 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:49,077 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:49,078 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:49,078 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:49,078 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 20:45:49,078 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D03628992BF34FA1A5488F67378A92C5, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:49,078 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 70DA86223CD7470C90C12B19E2874E00, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:49,078 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:49,079 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 70DA86223CD7470C90C12B19E2874E00
2016-05-17 20:45:49,079 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:49,079 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 70DA86223CD7470C90C12B19E2874E00
2016-05-17 20:45:49,079 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 70DA86223CD7470C90C12B19E2874E00 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:49,080 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:49,080 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:49,080 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:49,081 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517948800_0001_1_00_000000_0
2016-05-17 20:45:49,081 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:49,083 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1f6ed6d3
2016-05-17 20:45:49,084 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower+upper.txt"]
2016-05-17 20:45:49,084 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1f6ed6d3
2016-05-17 20:45:49,085 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:49,085 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:49,086 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower+upper.txt"] streamed: true, id: 70DA86223CD7470C90C12B19E2874E00
2016-05-17 20:45:49,086 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped"], id: D03628992BF34FA1A5488F67378A92C5
2016-05-17 20:45:49,087 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped"]
2016-05-17 20:45:49,087 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower+upper.txt"]
2016-05-17 20:45:49,088 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:49,088 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F8590CB2C4D643849371443AE7B4080B, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:49,091 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:49,092 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:49,092 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517948800_0001_1_00_000000_0
2016-05-17 20:45:49,093 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517948800_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 20:45:49,094 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517948800_0001_1_00_000000_0
2016-05-17 20:45:49,094 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F8590CB2C4D643849371443AE7B4080B, mem on close (mb), free: 158, total: 461, max: 672
2016-05-17 20:45:49 Completed running task attempt: attempt_1463517948800_0001_1_00_000000_0
2016-05-17 20:45:49,096 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517948800_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:49,099 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:49,099 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F8590CB2C4D643849371443AE7B4080B, taskAttemptId=attempt_1463517948800_0001_1_00_000000_0, startTime=1463517949049, finishTime=1463517949099, timeTaken=50, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463517949088, Process_Duration=6, Process_End_Time=1463517949094, Read_Duration=2, Tuples_Read=5, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10
2016-05-17 20:45:49,099 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517948800_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:49,100 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517948800_0001_1_00_000000_0
2016-05-17 20:45:49,100 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:49,101 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:TASK_FINISHED]: vertexName=F8590CB2C4D643849371443AE7B4080B, taskId=task_1463517948800_0001_1_00_000000, startTime=1463517949049, finishTime=1463517949100, timeTaken=51, status=SUCCEEDED, successfulAttemptID=attempt_1463517948800_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463517949088, Process_Duration=6, Process_End_Time=1463517949094, Read_Duration=2, Tuples_Read=5, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10
2016-05-17 20:45:49,101 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517948800_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:49,101 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B] : 1
2016-05-17 20:45:49,101 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B]
2016-05-17 20:45:49,101 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:49,102 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:49,103 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:VERTEX_FINISHED]: vertexName=F8590CB2C4D643849371443AE7B4080B, vertexId=vertex_1463517948800_0001_1_00, initRequestedTime=1463517949007, initedTime=1463517949028, startRequestedTime=1463517949029, startedTime=1463517949029, finishTime=1463517949101, timeTaken=72, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463517949088, Process_Duration=6, Process_End_Time=1463517949094, Read_Duration=2, Tuples_Read=5, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10, vertexStats=firstTaskStartTime=1463517949049, firstTasksToStart=[ task_1463517948800_0001_1_00_000000 ], lastTaskFinishTime=1463517949099, lastTasksToFinish=[ task_1463517948800_0001_1_00_000000 ], minTaskDuration=50, maxTaskDuration=50, avgTaskDuration=50.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517948800_0001_1_00_000000 ], longestDurationTasks=[ task_1463517948800_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:49,103 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:49,104 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10][cascading.flow.SliceCounters Process_Begin_Time=1463517949088, Process_Duration=6, Process_End_Time=1463517949094, Read_Duration=2, Tuples_Read=5, Tuples_Written=10, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=5, Tuples_Written=10]]
2016-05-17 20:45:49,104 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517948800_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:49,104 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:49,105 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:49,105 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517948800_0001_00_000001, stoppedTime=1463517949105, exitStatus=0
2016-05-17 20:45:49,105 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:49,105 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517948800_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:49,105 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 20:45:49,106 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517948800_0001_1
2016-05-17 20:45:49,106 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517948800_0001_1
2016-05-17 20:45:49,106 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: D03628992BF34FA1A5488F67378A92C5 for vertex: vertex_1463517948800_0001_1_00 [F8590CB2C4D643849371443AE7B4080B]
2016-05-17 20:45:49,106 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517948800_0001_00_000001
2016-05-17 20:45:49,107 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517948800_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:49,107 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517948800_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:49,108 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517948800_0001_00_000001, Exiting
2016-05-17 20:45:49,108 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517948800_0001_00_000001
2016-05-17 20:45:49,108 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517948800_0001_00_000001
2016-05-17 20:45:49,112 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517948800_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:49,112 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517948800_0001_00_000001 completed successfully
2016-05-17 20:45:49,118 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517948800_0001_1, queueSize=0
2016-05-17 20:45:49,119 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:49,119 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517948800_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517948800_0001_1, startTime=1463517949007, finishTime=1463517949117, timeTaken=110, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463517949088, Process_Duration=6, Process_End_Time=1463517949094, Read_Duration=2, Tuples_Read=5, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10
2016-05-17 20:45:49,119 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517948800_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:49,119 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517948800_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:49,119 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517948800_0001_00_000001
2016-05-17 20:45:49,119 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517948800_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:49 Completed Dag: dag_1463517948800_0001_1
2016-05-17 20:45:49,120 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517948800_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:49,120 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517948800_0001_1_post
2016-05-17 20:45:49,120 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:49,120 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:49,133 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...orm/testUnGroup/ungrouped, applicationId=application_1463517948800_0001
2016-05-17 20:45:49,136 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:49,136 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:49,136 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:49,136 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:49,137 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:49,137 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:49,137 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:49,138 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:49,138 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:49,138 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:49,139 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:49,139 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:49,140 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:49,144 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/47573793A067401B91C8E169DCD8A63D/DAB5F219900741AC80176FC89FD7907E/.tez/application_1463517948800_0001
2016-05-17 20:45:49,145 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped/_temporary
2016-05-17 20:45:49,146 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.369
2016-05-17 20:45:49,182 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped
2016-05-17 20:45:49,213 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:49,213 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:49,271 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.028
2016-05-17 20:45:49,286 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.043
2016-05-17 20:45:49,287 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-17 20:45:49,287 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-17 20:45:49,288 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:49,306 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt
2016-05-17 20:45:49,320 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt
2016-05-17 20:45:49,328 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown
2016-05-17 20:45:49,427 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] starting
2016-05-17 20:45:49,428 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-05-17 20:45:49,429 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 20:45:49,430 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown"]
2016-05-17 20:45:49,430 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] parallel execution of steps is enabled: false
2016-05-17 20:45:49,430 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executing total steps: 1
2016-05-17 20:45:49,430 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] allocating management threads: 1
2016-05-17 20:45:49,431 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] starting step: (1/1) ...lexcogrouprelativeunknown
2016-05-17 20:45:49,431 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [lower*upper] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:49,447 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tez session mode enabled: true
2016-05-17 20:45:49,452 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:49,453 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:49,453 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:49,463 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462/.tez/application_1463517949453_0001 doesn't exist and is created
2016-05-17 20:45:49,480 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:49,480 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:49,481 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462/.tez/application_1463517949453_0001
2016-05-17 20:45:49,486 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517949453_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:49,498 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:49,499 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:49,502 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:49,502 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:49,502 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462/.tez/application_1463517949453_0001/localmode-log-dir
2016-05-17 20:45:49,502 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462/.tez/application_1463517949453_0001/localmode-log-dir/history.txt.appattempt_1463517949453_0001_000000, maxErrors=10
2016-05-17 20:45:49,502 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:49,503 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517949453_0001_000000, appSubmitTime=1463517949485, launchTime=1463517949486
2016-05-17 20:45:49,508 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:49,509 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:49,511 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:49,511 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:49,513 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:50928
2016-05-17 20:45:49,515 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:49,517 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517949453_0001_000000, startTime=1463517949517
2016-05-17 20:45:49,517 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:49,517 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:49,580 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:49,580 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:49,580 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lexcogrouprelativeunknown, applicationId=application_1463517949453_0001, dagName=[5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-05-17 20:45:49,598 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-05-17 20:45:49,600 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517949453_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462/.tez/application_1463517949453_0001/localmode-log-dir/dag_1463517949453_0001_1.dot
2016-05-17 20:45:49,602 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462/.tez/application_1463517949453_0001/localmode-log-dir/dag_1463517949453_0001_1-tez-dag.pb.txt
2016-05-17 20:45:49,630 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517949453_0001_1
2016-05-17 20:45:49,630 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:49,631 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-05-17 20:45:49 Running Dag: dag_1463517949453_0001_1
2016-05-17 20:45:49,643 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517949453_0001_1, submitTime=1463517949598
2016-05-17 20:45:49,643 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:49,644 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex CFD26EBBC047461FA4A780DCF2EDE82F
2016-05-17 20:45:49,646 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:49,647 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:49,647 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517949453_0001_1, initTime=1463517949643
2016-05-17 20:45:49,647 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517949453_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:49,648 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:DAG_STARTED]: dagID=dag_1463517949453_0001_1, startTime=1463517949648
2016-05-17 20:45:49,648 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:49,649 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:49,649 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517949453_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:49,649 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D5951A42F5F949D9AD7CF3972EB9E22E : {F7A74F1017004C2E896FE02CA641E7F9={InputName=F7A74F1017004C2E896FE02CA641E7F9}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:49,649 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: F7A74F1017004C2E896FE02CA641E7F9, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:49,649 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E]
2016-05-17 20:45:49,650 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E]
2016-05-17 20:45:49,650 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E]
2016-05-17 20:45:49,650 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E]. Starting root input initializers: 1
2016-05-17 20:45:49,651 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:49,651 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CDB30EBFD3D44B38AFF1F4B60286850E : {A154428B660F4B11BBAB47BE399BB520={InputName=A154428B660F4B11BBAB47BE399BB520}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:49,651 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: A154428B660F4B11BBAB47BE399BB520, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:49,651 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E]
2016-05-17 20:45:49,651 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E]
2016-05-17 20:45:49,651 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E]
2016-05-17 20:45:49,652 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E]. Starting root input initializers: 1
2016-05-17 20:45:49,652 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: F7A74F1017004C2E896FE02CA641E7F9 on vertex vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E]
2016-05-17 20:45:49,653 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:49,654 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input F7A74F1017004C2E896FE02CA641E7F9 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:49,654 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:49,654 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:49,656 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-05-17 20:45:49,656 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:49,657 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F]
2016-05-17 20:45:49,657 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: F7A74F1017004C2E896FE02CA641E7F9 on vertex vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E]
2016-05-17 20:45:49,657 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lexcogrouprelativeunknown, applicationId=application_1463517949453_0001, dagName=[5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-05-17 20:45:49,660 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 20:45:49,660 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F]
2016-05-17 20:45:49,660 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F]
2016-05-17 20:45:49,660 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F]
2016-05-17 20:45:49,660 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: A154428B660F4B11BBAB47BE399BB520 on vertex vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E]
2016-05-17 20:45:49,660 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=4A4B6074FD794C7F807C3AC2D233A01D, vertexId=vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:49,662 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=4A4B6074FD794C7F807C3AC2D233A01D, vertexId=vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F]
2016-05-17 20:45:49,664 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=4A4B6074FD794C7F807C3AC2D233A01D, vertexName=CFD26EBBC047461FA4A780DCF2EDE82F, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:49,664 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=4A4B6074FD794C7F807C3AC2D233A01D, vertexId=vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F]
2016-05-17 20:45:49,666 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:49,666 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:49,666 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input A154428B660F4B11BBAB47BE399BB520 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:49,667 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CFD26EBBC047461FA4A780DCF2EDE82F, vertexId=vertex_1463517949453_0001_1_02, initRequestedTime=1463517949657, initedTime=1463517949666, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 20:45:49,667 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F] transitioned from NEW to INITED due to event V_INIT
2016-05-17 20:45:49,667 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:49,668 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {F7A74F1017004C2E896FE02CA641E7F9=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:49,668 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E] parallelism set to 1
2016-05-17 20:45:49,669 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:49,670 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517949453_0001_1_00, eventCount=1
2016-05-17 20:45:49,671 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:49,671 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D5951A42F5F949D9AD7CF3972EB9E22E, vertexId=vertex_1463517949453_0001_1_00, initRequestedTime=1463517949649, initedTime=1463517949671, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:49,672 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E] with distanceFromRoot: 0
2016-05-17 20:45:49,672 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:49,672 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D5951A42F5F949D9AD7CF3972EB9E22E
2016-05-17 20:45:49,673 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E]
2016-05-17 20:45:49,673 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:49,673 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517949453_0001_1_00, startRequestedTime=1463517949672, startedTime=1463517949672
2016-05-17 20:45:49,674 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:49,674 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:49,671 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:49,675 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:49,675 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:TASK_STARTED]: vertexName=D5951A42F5F949D9AD7CF3972EB9E22E, taskId=task_1463517949453_0001_1_00_000000, scheduledTime=1463517949674, launchTime=1463517949674
2016-05-17 20:45:49,676 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517949453_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:49,676 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463517949453_0001_1_00 for vertex: vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F] numStartedSources: 1 numSources: 2
2016-05-17 20:45:49,676 INFO impl.VertexImpl (VertexImpl.java:transition(3160)) - Cannot start vertex: vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F] numStartedSources: 1 numSources: 2
2016-05-17 20:45:49,677 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E]
2016-05-17 20:45:49,677 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: D5951A42F5F949D9AD7CF3972EB9E22E, VertexParallelism: 1, TaskAttemptID:attempt_1463517949453_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F7A74F1017004C2E896FE02CA641E7F9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CFD26EBBC047461FA4A780DCF2EDE82F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:49,678 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517949453_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:49,678 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:49,679 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:49,677 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-05-17 20:45:49,680 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:49,682 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:49,683 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517949453_0001_00_000001
2016-05-17 20:45:49,683 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:49,684 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: A154428B660F4B11BBAB47BE399BB520 on vertex vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E]
2016-05-17 20:45:49,685 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:49,685 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {A154428B660F4B11BBAB47BE399BB520=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:49,685 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517949453_0001_00_000001
2016-05-17 20:45:49,685 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E] parallelism set to 1
2016-05-17 20:45:49,685 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:49,686 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:49,686 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517949453_0001_1_01, eventCount=1
2016-05-17 20:45:49,689 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CDB30EBFD3D44B38AFF1F4B60286850E, vertexId=vertex_1463517949453_0001_1_01, initRequestedTime=1463517949651, initedTime=1463517949689, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:49,689 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E] with distanceFromRoot: 0
2016-05-17 20:45:49,689 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:49,689 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CDB30EBFD3D44B38AFF1F4B60286850E
2016-05-17 20:45:49,689 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E]
2016-05-17 20:45:49,689 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517949453_0001_00_000001, launchTime=1463517949689
2016-05-17 20:45:49,690 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:49,690 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:49,690 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517949453_0001_1_01, startRequestedTime=1463517949689, startedTime=1463517949689
2016-05-17 20:45:49,690 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:49,690 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:49,691 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:49,691 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517949453_0001_00_000001
2016-05-17 20:45:49,691 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:49,691 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:TASK_STARTED]: vertexName=CDB30EBFD3D44B38AFF1F4B60286850E, taskId=task_1463517949453_0001_1_01_000000, scheduledTime=1463517949691, launchTime=1463517949691
2016-05-17 20:45:49,694 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517949453_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:49,694 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] submitted tez dag to app master: application_1463517949453_0001, with dag id: dag_1463517949453_0001_1
2016-05-17 20:45:49,692 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517949453_0001_1_00_000000_0] to container: [container_1463517949453_0001_00_000001]
2016-05-17 20:45:49,694 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463517949453_0001_1_01 for vertex: vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F] numStartedSources: 2 numSources: 2
2016-05-17 20:45:49,695 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:49,695 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F] with distanceFromRoot: 1
2016-05-17 20:45:49,695 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517949453_0001_00_000001 given task: attempt_1463517949453_0001_1_00_000000_0
2016-05-17 20:45:49,695 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E]
2016-05-17 20:45:49,695 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517949453_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517949453_0001_1_00_000000_0
2016-05-17 20:45:49,697 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517949453_0001_1_00_000000_0] started. Is using containerId: [container_1463517949453_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:49 Starting to run new task attempt: attempt_1463517949453_0001_1_00_000000_0
2016-05-17 20:45:49,697 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517949453_0001_1_00_000000_0
2016-05-17 20:45:49,697 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D5951A42F5F949D9AD7CF3972EB9E22E, taskAttemptId=attempt_1463517949453_0001_1_00_000000_0, startTime=1463517949696, containerId=container_1463517949453_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517949453_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:49,697 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517949453_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:49,697 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:49,698 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: CDB30EBFD3D44B38AFF1F4B60286850E in vertex: CFD26EBBC047461FA4A780DCF2EDE82F
2016-05-17 20:45:49,698 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:49,698 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D5951A42F5F949D9AD7CF3972EB9E22E in vertex: CFD26EBBC047461FA4A780DCF2EDE82F
2016-05-17 20:45:49,698 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:49,698 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: CFD26EBBC047461FA4A780DCF2EDE82F with 2 source tasks and 1 pending tasks
2016-05-17 20:45:49,698 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:49,699 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517949453_0001_1_02, startRequestedTime=1463517949697, startedTime=1463517949697
2016-05-17 20:45:49,699 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:49,699 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:49,699 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:49,699 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:49,699 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: D5951A42F5F949D9AD7CF3972EB9E22E, VertexParallelism: 1, TaskAttemptID:attempt_1463517949453_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F7A74F1017004C2E896FE02CA641E7F9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CFD26EBBC047461FA4A780DCF2EDE82F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:49,700 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:49,699 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:49,707 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517949453_0001_1_00_000000_0
2016-05-17 20:45:49,707 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:49,707 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:49,708 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F7A74F1017004C2E896FE02CA641E7F9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:49,708 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:49,710 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CFD26EBBC047461FA4A780DCF2EDE82F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 20:45:49,711 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:49,711 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CFD26EBBC047461FA4A780DCF2EDE82F
2016-05-17 20:45:49,713 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:49,714 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 20:45:49,714 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: CFD26EBBC047461FA4A780DCF2EDE82F
2016-05-17 20:45:49,720 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CFD26EBBC047461FA4A780DCF2EDE82F
2016-05-17 20:45:49,721 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517949453_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:49,720 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: CDB30EBFD3D44B38AFF1F4B60286850E, VertexParallelism: 1, TaskAttemptID:attempt_1463517949453_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A154428B660F4B11BBAB47BE399BB520, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CFD26EBBC047461FA4A780DCF2EDE82F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:49,721 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517949453_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:49,722 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517949453_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:49,722 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:49,720 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:49,727 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F7A74F1017004C2E896FE02CA641E7F9
2016-05-17 20:45:49,728 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: F7A74F1017004C2E896FE02CA641E7F9
2016-05-17 20:45:49,728 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517949453_0001_00_000002
2016-05-17 20:45:49,728 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:49,732 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517949453_0001_00_000002
2016-05-17 20:45:49,732 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:49,737 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D5951A42F5F949D9AD7CF3972EB9E22E, ordinal: 1
2016-05-17 20:45:49,737 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D5951A42F5F949D9AD7CF3972EB9E22E, mem on start (mb), free: 46, total: 461, max: 672
2016-05-17 20:45:49,737 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:49,738 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:49,738 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:49,743 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:49,743 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: F7A74F1017004C2E896FE02CA641E7F9
2016-05-17 20:45:49,743 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F7A74F1017004C2E896FE02CA641E7F9
2016-05-17 20:45:49,744 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:49,752 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:49,752 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:49,753 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517949453_0001_00_000002, launchTime=1463517949752
2016-05-17 20:45:49,754 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:49,755 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 20:45:49,755 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:49,756 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:49,756 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:49,757 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 20:45:49,757 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CFD26EBBC047461FA4A780DCF2EDE82F, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 20:45:49,758 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F7A74F1017004C2E896FE02CA641E7F9, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:49,758 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:49,759 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F7A74F1017004C2E896FE02CA641E7F9
2016-05-17 20:45:49,759 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:49,759 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F7A74F1017004C2E896FE02CA641E7F9
2016-05-17 20:45:49,759 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517949453_0001_00_000002
2016-05-17 20:45:49,761 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F7A74F1017004C2E896FE02CA641E7F9 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:49,762 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:49,762 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:49,764 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:49,763 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517949453_0001_1_01_000000_0] to container: [container_1463517949453_0001_00_000002]
2016-05-17 20:45:49,764 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:49,765 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517949453_0001_1_01_000000_0] started. Is using containerId: [container_1463517949453_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 20:45:49,765 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CDB30EBFD3D44B38AFF1F4B60286850E, taskAttemptId=attempt_1463517949453_0001_1_01_000000_0, startTime=1463517949765, containerId=container_1463517949453_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517949453_0001_00_000002/teamcity, completedLogs=
2016-05-17 20:45:49,766 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:49,766 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517949453_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:49,766 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517949453_0001_00_000002 given task: attempt_1463517949453_0001_1_01_000000_0
2016-05-17 20:45:49,766 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517949453_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517949453_0001_1_01_000000_0
2016-05-17 20:45:49,766 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517949453_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:49 Starting to run new task attempt: attempt_1463517949453_0001_1_01_000000_0
2016-05-17 20:45:49,767 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517949453_0001_1_01_000000_0
2016-05-17 20:45:49,768 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:49,768 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:49,768 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:49,768 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:49,768 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:49,768 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517949453_0001_1_00_000000_0
2016-05-17 20:45:49,768 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:49,772 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: CDB30EBFD3D44B38AFF1F4B60286850E, VertexParallelism: 1, TaskAttemptID:attempt_1463517949453_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A154428B660F4B11BBAB47BE399BB520, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CFD26EBBC047461FA4A780DCF2EDE82F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:49,772 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:49,772 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 20:45:49,776 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"] streamed: true, id: F7A74F1017004C2E896FE02CA641E7F9
2016-05-17 20:45:49,776 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]], id: 5F8FD08921AD47D1B0A054A29A12F241
2016-05-17 20:45:49,776 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] 5F8FD08921AD47D1B0A054A29A12F241
2016-05-17 20:45:49,778 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:49,778 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517949453_0001_1_01_000000_0
2016-05-17 20:45:49,779 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:49,779 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:49,779 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2c32d8b8
2016-05-17 20:45:49,786 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A154428B660F4B11BBAB47BE399BB520, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:49,787 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:49,792 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CFD26EBBC047461FA4A780DCF2EDE82F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 20:45:49,793 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517949453_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 20:45:49,786 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@1dddf775; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@57a131a4; comparator=cascading.tuple.hadoop.util.TupleComparator@536fe809; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 20:45:49,793 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 20:45:49,794 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 20:45:49,794 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:49,793 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:49,831 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CFD26EBBC047461FA4A780DCF2EDE82F
2016-05-17 20:45:49,792 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A154428B660F4B11BBAB47BE399BB520
2016-05-17 20:45:49,831 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: A154428B660F4B11BBAB47BE399BB520
2016-05-17 20:45:49,833 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 20:45:49,833 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: CFD26EBBC047461FA4A780DCF2EDE82F
2016-05-17 20:45:49,787 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:49,787 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2c32d8b8
2016-05-17 20:45:49,834 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:49,834 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:49,844 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 20:45:49,844 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:49,845 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: A154428B660F4B11BBAB47BE399BB520
2016-05-17 20:45:49,845 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A154428B660F4B11BBAB47BE399BB520
2016-05-17 20:45:49,844 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CFD26EBBC047461FA4A780DCF2EDE82F
2016-05-17 20:45:49,845 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 20:45:49,845 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 20:45:49,846 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 20:45:49,846 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 20:45:49,846 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:49,846 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 20:45:49,847 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:49,847 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D5951A42F5F949D9AD7CF3972EB9E22E, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:49,852 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517949453_0001_1_00_000000_0
2016-05-17 20:45:49,852 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D5951A42F5F949D9AD7CF3972EB9E22E, mem on close (mb), free: 136, total: 439, max: 672
2016-05-17 20:45:49,852 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 20:45:49,853 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 20:45:49,853 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-05-17 20:45:49,853 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-05-17 20:45:49,858 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 20:45:49,858 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CDB30EBFD3D44B38AFF1F4B60286850E, ordinal: 0
2016-05-17 20:45:49,859 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CDB30EBFD3D44B38AFF1F4B60286850E, mem on start (mb), free: 135, total: 439, max: 672
2016-05-17 20:45:49,859 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:49,859 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:49,859 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:49,860 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:49,860 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:49,860 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:49,860 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:49,860 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 20:45:49,861 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:49,861 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 20:45:49,861 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A154428B660F4B11BBAB47BE399BB520, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:49,861 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CFD26EBBC047461FA4A780DCF2EDE82F, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 20:45:49,861 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:49,861 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A154428B660F4B11BBAB47BE399BB520
2016-05-17 20:45:49,862 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:49,862 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A154428B660F4B11BBAB47BE399BB520
2016-05-17 20:45:49,862 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A154428B660F4B11BBAB47BE399BB520 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:49,862 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:49,862 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:49,863 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:49,863 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517949453_0001_1_01_000000_0
2016-05-17 20:45:49 Completed running task attempt: attempt_1463517949453_0001_1_00_000000_0
2016-05-17 20:45:49,867 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-05-17 20:45:49,867 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517949453_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:49,867 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt"] streamed: true, id: A154428B660F4B11BBAB47BE399BB520
2016-05-17 20:45:49,868 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]], id: 5F8FD08921AD47D1B0A054A29A12F241
2016-05-17 20:45:49,869 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517949453_0001_1_00, eventCount=1
2016-05-17 20:45:49,869 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] 5F8FD08921AD47D1B0A054A29A12F241
2016-05-17 20:45:49,869 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=35, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=460324864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463517949847, Process_Duration=5, Process_End_Time=1463517949852, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-17 20:45:49,869 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:49,870 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D5951A42F5F949D9AD7CF3972EB9E22E, taskAttemptId=attempt_1463517949453_0001_1_00_000000_0, startTime=1463517949696, finishTime=1463517949869, timeTaken=173, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=35, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=460324864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517949847, Process_Duration=5, Process_End_Time=1463517949852, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 20:45:49,870 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517949453_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:49,871 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517949453_0001_1_00_000000_0
2016-05-17 20:45:49,871 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:49,871 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:49,872 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:TASK_FINISHED]: vertexName=D5951A42F5F949D9AD7CF3972EB9E22E, taskId=task_1463517949453_0001_1_00_000000, startTime=1463517949696, finishTime=1463517949871, timeTaken=175, status=SUCCEEDED, successfulAttemptID=attempt_1463517949453_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=35, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=460324864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517949847, Process_Duration=5, Process_End_Time=1463517949852, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 20:45:49,872 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517949453_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:49,872 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E] : 1
2016-05-17 20:45:49,872 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E]
2016-05-17 20:45:49,873 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:49,873 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@424c8abf; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@2f29239a; comparator=cascading.tuple.hadoop.util.TupleComparator@35e06ba6; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 20:45:49,874 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:VERTEX_FINISHED]: vertexName=D5951A42F5F949D9AD7CF3972EB9E22E, vertexId=vertex_1463517949453_0001_1_00, initRequestedTime=1463517949649, initedTime=1463517949671, startRequestedTime=1463517949672, startedTime=1463517949672, finishTime=1463517949872, timeTaken=200, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=35, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=460324864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517949847, Process_Duration=5, Process_End_Time=1463517949852, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463517949696, firstTasksToStart=[ task_1463517949453_0001_1_00_000000 ], lastTaskFinishTime=1463517949869, lastTasksToFinish=[ task_1463517949453_0001_1_00_000000 ], minTaskDuration=173, maxTaskDuration=173, avgTaskDuration=173.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517949453_0001_1_00_000000 ], longestDurationTasks=[ task_1463517949453_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:49,875 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:49,875 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:49,874 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:49,875 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F] attempt: attempt_1463517949453_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 20:45:49,876 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517949453_0001_00_000001, stoppedTime=1463517949876, exitStatus=0
2016-05-17 20:45:49,876 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:49,876 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 20:45:49,876 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:49,876 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 20:45:49,877 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 20:45:49,877 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:49,873 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:49,873 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:49,876 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463517949453_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 20:45:49,882 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 20:45:49,883 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2cf1567a
2016-05-17 20:45:49,883 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517949453_0001_00_000001
2016-05-17 20:45:49,885 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517949453_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:49,885 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517949453_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:49,885 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517949453_0001_00_000001, Exiting
2016-05-17 20:45:49,885 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517949453_0001_00_000001
2016-05-17 20:45:49,886 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517949453_0001_00_000001
2016-05-17 20:45:49,885 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2cf1567a
2016-05-17 20:45:49,895 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:49,896 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:49,895 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517949453_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:49,894 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 20:45:49,896 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 20:45:49,897 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 20:45:49,897 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 20:45:49,897 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 20:45:49,897 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517949453_0001_00_000001 completed successfully
2016-05-17 20:45:49,897 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 20:45:49,900 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-05-17 20:45:49,900 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:49,900 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CDB30EBFD3D44B38AFF1F4B60286850E, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:49,904 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517949453_0001_1_01_000000_0
2016-05-17 20:45:49,905 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CDB30EBFD3D44B38AFF1F4B60286850E, mem on close (mb), free: 31, total: 439, max: 672
2016-05-17 20:45:49,905 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 20:45:49,906 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 20:45:49,906 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-05-17 20:45:49,906 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-05-17 20:45:49,910 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 20:45:49 Completed running task attempt: attempt_1463517949453_0001_1_01_000000_0
2016-05-17 20:45:49,916 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517949453_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:49,917 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517949453_0001_1_01, eventCount=1
2016-05-17 20:45:49,917 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=8155, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=35, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=460324864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463517949900, Process_Duration=4, Process_End_Time=1463517949904, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-17 20:45:49,918 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CDB30EBFD3D44B38AFF1F4B60286850E, taskAttemptId=attempt_1463517949453_0001_1_01_000000_0, startTime=1463517949765, finishTime=1463517949918, timeTaken=153, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=8155, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=35, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=460324864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517949900, Process_Duration=4, Process_End_Time=1463517949904, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 20:45:49,918 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517949453_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:49,919 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517949453_0001_1_01_000000_0
2016-05-17 20:45:49,920 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:TASK_FINISHED]: vertexName=CDB30EBFD3D44B38AFF1F4B60286850E, taskId=task_1463517949453_0001_1_01_000000, startTime=1463517949765, finishTime=1463517949919, timeTaken=154, status=SUCCEEDED, successfulAttemptID=attempt_1463517949453_0001_1_01_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=8155, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=35, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=460324864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517949900, Process_Duration=4, Process_End_Time=1463517949904, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 20:45:49,920 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517949453_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:49,920 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E] : 1
2016-05-17 20:45:49,920 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E]
2016-05-17 20:45:49,920 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:49,918 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:49,922 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:49,922 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:VERTEX_FINISHED]: vertexName=CDB30EBFD3D44B38AFF1F4B60286850E, vertexId=vertex_1463517949453_0001_1_01, initRequestedTime=1463517949651, initedTime=1463517949689, startRequestedTime=1463517949689, startedTime=1463517949689, finishTime=1463517949920, timeTaken=231, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=8155, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=35, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=460324864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517949900, Process_Duration=4, Process_End_Time=1463517949904, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463517949765, firstTasksToStart=[ task_1463517949453_0001_1_01_000000 ], lastTaskFinishTime=1463517949918, lastTasksToFinish=[ task_1463517949453_0001_1_01_000000 ], minTaskDuration=153, maxTaskDuration=153, avgTaskDuration=153.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517949453_0001_1_01_000000 ], longestDurationTasks=[ task_1463517949453_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:49,922 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:49,922 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F] attempt: attempt_1463517949453_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 20:45:49,922 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: CFD26EBBC047461FA4A780DCF2EDE82F
2016-05-17 20:45:49,922 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F]
2016-05-17 20:45:49,922 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:49,923 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:49,923 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517949453_0001_00_000002, stoppedTime=1463517949923, exitStatus=0
2016-05-17 20:45:49,923 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:49,923 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 20:45:49,924 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:49,924 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:TASK_STARTED]: vertexName=CFD26EBBC047461FA4A780DCF2EDE82F, taskId=task_1463517949453_0001_1_02_000000, scheduledTime=1463517949924, launchTime=1463517949924
2016-05-17 20:45:49,924 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:49,924 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:49,924 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:49,924 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517949453_0001_1_02_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:49,925 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F]
2016-05-17 20:45:49,925 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463517949453_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-17 20:45:49,925 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: CFD26EBBC047461FA4A780DCF2EDE82F, VertexParallelism: 1, TaskAttemptID:attempt_1463517949453_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D5951A42F5F949D9AD7CF3972EB9E22E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=CDB30EBFD3D44B38AFF1F4B60286850E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=4A4B6074FD794C7F807C3AC2D233A01D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:49,926 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517949453_0001_1_02_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:49,926 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 20:45:49,926 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:49,926 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517949453_0001_00_000002
2016-05-17 20:45:50,128 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517949453_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 20:45:50,128 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517949453_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:50,130 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517949453_0001_00_000002, Exiting
2016-05-17 20:45:50,130 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517949453_0001_00_000002
2016-05-17 20:45:50,130 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517949453_0001_00_000002
2016-05-17 20:45:50,132 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517949453_0001_00_000003
2016-05-17 20:45:50,132 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000003 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:50,132 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517949453_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:50,132 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517949453_0001_00_000002 completed successfully
2016-05-17 20:45:50,132 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 20:45:50,134 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517949453_0001_00_000003
2016-05-17 20:45:50,135 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 20:45:50,142 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000003 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:50,142 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:50,142 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517949453_0001_00_000003, launchTime=1463517950142
2016-05-17 20:45:50,143 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517949453_0001_00_000003
2016-05-17 20:45:50,144 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517949453_0001_1_02_000000_0] to container: [container_1463517949453_0001_00_000003]
2016-05-17 20:45:50,144 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000003 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:50,145 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517949453_0001_00_000003 given task: attempt_1463517949453_0001_1_02_000000_0
2016-05-17 20:45:50,145 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517949453_0001_1_02_000000_0] started. Is using containerId: [container_1463517949453_0001_00_000003] on NM: [127.0.0.1:0]
2016-05-17 20:45:50,145 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CFD26EBBC047461FA4A780DCF2EDE82F, taskAttemptId=attempt_1463517949453_0001_1_02_000000_0, startTime=1463517950145, containerId=container_1463517949453_0001_00_000003, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517949453_0001_00_000003/teamcity, completedLogs=
2016-05-17 20:45:50,145 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517949453_0001_1_02_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:50,146 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517949453_0001_1_02_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:50,145 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517949453_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517949453_0001_1_02_000000_0
2016-05-17 20:45:50,145 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:50 Starting to run new task attempt: attempt_1463517949453_0001_1_02_000000_0
2016-05-17 20:45:50,147 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517949453_0001_1_02_000000_0
2016-05-17 20:45:50,147 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:50,148 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:50,148 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:50,148 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:50,148 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:50,148 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:50,148 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: CFD26EBBC047461FA4A780DCF2EDE82F, VertexParallelism: 1, TaskAttemptID:attempt_1463517949453_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D5951A42F5F949D9AD7CF3972EB9E22E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=CDB30EBFD3D44B38AFF1F4B60286850E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=4A4B6074FD794C7F807C3AC2D233A01D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:50,148 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:50,152 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517949453_0001_1_02_000000_0
2016-05-17 20:45:50,152 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:50,153 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:50,152 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517949453_0001_1_02_000000_0 sent events: (0-2)
2016-05-17 20:45:50,154 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D5951A42F5F949D9AD7CF3972EB9E22E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 20:45:50,155 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:50,155 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:50,155 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=CDB30EBFD3D44B38AFF1F4B60286850E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 20:45:50,156 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:50,156 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: CDB30EBFD3D44B38AFF1F4B60286850E
2016-05-17 20:45:50,158 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=4A4B6074FD794C7F807C3AC2D233A01D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:50,158 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:50,158 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 20:45:50,159 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 20:45:50,159 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: CDB30EBFD3D44B38AFF1F4B60286850E
2016-05-17 20:45:50,160 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: CDB30EBFD3D44B38AFF1F4B60286850E
2016-05-17 20:45:50,159 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D5951A42F5F949D9AD7CF3972EB9E22E
2016-05-17 20:45:50,159 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:50,161 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 4A4B6074FD794C7F807C3AC2D233A01D
2016-05-17 20:45:50,162 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:50,162 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 4A4B6074FD794C7F807C3AC2D233A01D
2016-05-17 20:45:50,166 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 20:45:50,167 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 20:45:50,167 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D5951A42F5F949D9AD7CF3972EB9E22E
2016-05-17 20:45:50,168 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D5951A42F5F949D9AD7CF3972EB9E22E
2016-05-17 20:45:50,173 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CFD26EBBC047461FA4A780DCF2EDE82F, ordinal: 2
2016-05-17 20:45:50,173 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CFD26EBBC047461FA4A780DCF2EDE82F, mem on start (mb), free: 311, total: 333, max: 672
2016-05-17 20:45:50,173 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:50,174 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 3 initializers to finish
2016-05-17 20:45:50,174 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:50,175 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:50,178 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:50,178 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 4A4B6074FD794C7F807C3AC2D233A01D
2016-05-17 20:45:50,178 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:50,178 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:50,179 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 20:45:50,180 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:50,180 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:50,180 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 20:45:50,180 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345
2016-05-17 20:45:50,181 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 1268357504, TotalRequestedScaled: 6.0881160192E8, TotalJVMHeap: 704643072, TotalAvailable: 461541212, TotalRequested/TotalJVMHeap:1.80
2016-05-17 20:45:50,181 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, CDB30EBFD3D44B38AFF1F4B60286850E, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-05-17 20:45:50,181 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 4A4B6074FD794C7F807C3AC2D233A01D, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:50,181 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D5951A42F5F949D9AD7CF3972EB9E22E, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-05-17 20:45:50,182 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:50,182 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D5951A42F5F949D9AD7CF3972EB9E22E
2016-05-17 20:45:50,183 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:50,183 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 20:45:50,184 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D5951A42F5F949D9AD7CF3972EB9E22E with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 20:45:50,184 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-05-17 20:45:50,184 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 20:45:50,184 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 20:45:50,187 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D5951A42F5F949D9AD7CF3972EB9E22E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:50,187 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: CDB30EBFD3D44B38AFF1F4B60286850E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:50,187 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 2
2016-05-17 20:45:50,187 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 2 IOs to start
2016-05-17 20:45:50,187 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D5951A42F5F949D9AD7CF3972EB9E22E
2016-05-17 20:45:50,187 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: CDB30EBFD3D44B38AFF1F4B60286850E
2016-05-17 20:45:50,188 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:50,189 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:50,189 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 20:45:50,189 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: CDB30EBFD3D44B38AFF1F4B60286850E with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 20:45:50,189 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-05-17 20:45:50,189 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 20:45:50,189 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 20:45:50,191 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: CDB30EBFD3D44B38AFF1F4B60286850E
2016-05-17 20:45:50,191 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:50,196 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517949453_0001_1_02_000000_0
2016-05-17 20:45:50,201 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]]
2016-05-17 20:45:50,202 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] streamed: true, id: 5F8FD08921AD47D1B0A054A29A12F241
2016-05-17 20:45:50,202 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown"], id: 4A4B6074FD794C7F807C3AC2D233A01D
2016-05-17 20:45:50,202 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown"]
2016-05-17 20:45:50,202 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] 5F8FD08921AD47D1B0A054A29A12F241, for 2 inputs
2016-05-17 20:45:50,203 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] 5F8FD08921AD47D1B0A054A29A12F241, for 2 inputs
2016-05-17 20:45:50,211 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463517949453_0001_1_00_000000_0_10023, runDuration: 152804, ]
2016-05-17 20:45:50,213 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463517949453_0001_1_00_000000_0_10023] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1463517950213, TimeTaken=1, Rate=0.09 MB/s
2016-05-17 20:45:50,213 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D5951A42F5F949D9AD7CF3972EB9E22E
2016-05-17 20:45:50,213 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 20:45:50,213 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [D5951A42F5F949D9AD7CF3972EB9E22E] #4 in 1ms
2016-05-17 20:45:50,214 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D5951A42F5F949D9AD7CF3972EB9E22E] #4, status:false, isInterrupted:false
2016-05-17 20:45:50,214 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463517949453_0001_1_01_000000_0_10027, runDuration: 82522, ]
2016-05-17 20:45:50,215 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463517949453_0001_1_01_000000_0_10027] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1463517950215, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 20:45:50,216 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : CDB30EBFD3D44B38AFF1F4B60286850E
2016-05-17 20:45:50,216 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 20:45:50,216 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [CDB30EBFD3D44B38AFF1F4B60286850E] #5 in 2ms
2016-05-17 20:45:50,216 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 20:45:50,217 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-05-17 20:45:50,218 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:50,218 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 20:45:50,218 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-17 20:45:50,218 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D5951A42F5F949D9AD7CF3972EB9E22E
2016-05-17 20:45:50,219 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [CDB30EBFD3D44B38AFF1F4B60286850E] #5, status:false, isInterrupted:false
2016-05-17 20:45:50,219 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 20:45:50,220 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 20:45:50,220 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(89)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-17 20:45:50,220 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(94)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-17 20:45:50,221 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-05-17 20:45:50,221 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:50,221 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 20:45:50,221 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-17 20:45:50,222 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : CDB30EBFD3D44B38AFF1F4B60286850E
2016-05-17 20:45:50,222 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 20:45:50,226 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CFD26EBBC047461FA4A780DCF2EDE82F, all 2 inputs ready in: 00:00:00.000
2016-05-17 20:45:50,228 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@904c609; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@18657d7f
2016-05-17 20:45:50,230 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@2f12295c; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@51596f28
2016-05-17 20:45:50,234 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:50,235 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:50,235 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517949453_0001_1_02_000000_0
2016-05-17 20:45:50,237 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517949453_0001_1_02_000000_0 given a go for committing the task output.
2016-05-17 20:45:50,238 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517949453_0001_1_02_000000_0
2016-05-17 20:45:50,238 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CFD26EBBC047461FA4A780DCF2EDE82F, mem on close (mb), free: 305, total: 333, max: 672
2016-05-17 20:45:50,238 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D5951A42F5F949D9AD7CF3972EB9E22E
2016-05-17 20:45:50,239 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 20:45:50,240 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 20:45:50,240 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:50,241 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 20:45:50,241 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 20:45:50,241 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: CDB30EBFD3D44B38AFF1F4B60286850E
2016-05-17 20:45:50,242 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 20:45:50,243 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 20:45:50,244 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:50,244 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 20:45:50,244 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 20:45:50 Completed running task attempt: attempt_1463517949453_0001_1_02_000000_0
2016-05-17 20:45:50,246 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517949453_0001_1_02_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:50,247 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=349700096, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=27, MERGE_PHASE_TIME=33, FIRST_EVENT_RECEIVED=31, LAST_EVENT_RECEIVED=25][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463517950226, Process_Duration=12, Process_End_Time=1463517950238, Read_Duration=2, Tuples_Read=10, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-05-17 20:45:50,247 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:50,248 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CFD26EBBC047461FA4A780DCF2EDE82F, taskAttemptId=attempt_1463517949453_0001_1_02_000000_0, startTime=1463517950145, finishTime=1463517950247, timeTaken=102, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=349700096, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=27, MERGE_PHASE_TIME=33, FIRST_EVENT_RECEIVED=31, LAST_EVENT_RECEIVED=25, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517950226, Process_Duration=12, Process_End_Time=1463517950238, Read_Duration=2, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-17 20:45:50,248 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517949453_0001_1_02_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:50,248 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:50,248 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517949453_0001_1_02_000000_0
2016-05-17 20:45:50,249 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:50,250 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:TASK_FINISHED]: vertexName=CFD26EBBC047461FA4A780DCF2EDE82F, taskId=task_1463517949453_0001_1_02_000000, startTime=1463517950145, finishTime=1463517950249, timeTaken=104, status=SUCCEEDED, successfulAttemptID=attempt_1463517949453_0001_1_02_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=349700096, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=27, MERGE_PHASE_TIME=33, FIRST_EVENT_RECEIVED=31, LAST_EVENT_RECEIVED=25, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517950226, Process_Duration=12, Process_End_Time=1463517950238, Read_Duration=2, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-17 20:45:50,250 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517949453_0001_1_02_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:50,250 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F] : 1
2016-05-17 20:45:50,250 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F]
2016-05-17 20:45:50,251 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:50,251 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:50,252 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517949453_0001_00_000003
2016-05-17 20:45:50,252 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:50,253 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:VERTEX_FINISHED]: vertexName=CFD26EBBC047461FA4A780DCF2EDE82F, vertexId=vertex_1463517949453_0001_1_02, initRequestedTime=1463517949657, initedTime=1463517949666, startRequestedTime=1463517949697, startedTime=1463517949697, finishTime=1463517950250, timeTaken=553, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=349700096, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=27, MERGE_PHASE_TIME=33, FIRST_EVENT_RECEIVED=31, LAST_EVENT_RECEIVED=25, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517950226, Process_Duration=12, Process_End_Time=1463517950238, Read_Duration=2, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1463517950145, firstTasksToStart=[ task_1463517949453_0001_1_02_000000 ], lastTaskFinishTime=1463517950247, lastTasksToFinish=[ task_1463517949453_0001_1_02_000000 ], minTaskDuration=102, maxTaskDuration=102, avgTaskDuration=102.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517949453_0001_1_02_000000 ], longestDurationTasks=[ task_1463517949453_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:50,253 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:50,253 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000003 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:50,253 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517949453_0001_00_000003, stoppedTime=1463517950253, exitStatus=0
2016-05-17 20:45:50,253 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:50,254 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000003 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:50,254 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 20:45:50,254 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517949453_0001_1
2016-05-17 20:45:50,254 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517949453_0001_1
2016-05-17 20:45:50,255 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463517949453_0001_1_00 [D5951A42F5F949D9AD7CF3972EB9E22E]
2016-05-17 20:45:50,255 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463517949453_0001_1_01 [CDB30EBFD3D44B38AFF1F4B60286850E]
2016-05-17 20:45:50,255 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 4A4B6074FD794C7F807C3AC2D233A01D for vertex: vertex_1463517949453_0001_1_02 [CFD26EBBC047461FA4A780DCF2EDE82F]
2016-05-17 20:45:50,256 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517949453_0001_00_000003 is valid, but no longer registered, and will be killed
2016-05-17 20:45:50,256 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517949453_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:50,256 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517949453_0001_00_000003, Exiting
2016-05-17 20:45:50,256 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517949453_0001_00_000003
2016-05-17 20:45:50,256 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517949453_0001_00_000003
2016-05-17 20:45:50,263 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517949453_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:50,263 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517949453_0001_00_000003 completed successfully
2016-05-17 20:45:50,273 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517949453_0001_1, queueSize=0
2016-05-17 20:45:50,274 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:50,278 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517949453_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517949453_0001_1, startTime=1463517949648, finishTime=1463517950270, timeTaken=622, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=2, File System Counters, FILE_BYTES_READ=440, FILE_BYTES_WRITTEN=8334, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=70, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1270349824, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=15, OUTPUT_BYTES=150, OUTPUT_BYTES_WITH_OVERHEAD=182, OUTPUT_BYTES_PHYSICAL=190, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=27, MERGE_PHASE_TIME=33, FIRST_EVENT_RECEIVED=31, LAST_EVENT_RECEIVED=25, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4390553849973, Process_Duration=21, Process_End_Time=4390553849994, Read_Duration=8, Tuples_Read=20, Tuples_Written=15, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=5
2016-05-17 20:45:50,279 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517949453_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:50,279 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517949453_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:50,280 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517949453_0001_00_000003
2016-05-17 20:45:50,280 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517949453_0001_00_000003 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:50 Completed Dag: dag_1463517949453_0001_1
2016-05-17 20:45:50,282 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517949453_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:50,283 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517949453_0001_1_post
2016-05-17 20:45:50,283 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:50,283 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:50,288 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lexcogrouprelativeunknown, applicationId=application_1463517949453_0001
2016-05-17 20:45:50,290 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:50,290 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:50,290 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:50,290 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:50,290 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:50,291 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:50,292 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:50,292 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:50,292 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:50,292 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:50,293 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:50,293 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:50,293 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:50,298 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5936E21C7C274A63A02F866833090A06/DBF19813DD96493D99C036C1DE91D462/.tez/application_1463517949453_0001
2016-05-17 20:45:50,299 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown/_temporary
2016-05-17 20:45:50,300 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] completed in: 00:00.872
2016-05-17 20:45:50,374 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown
2016-05-17 20:45:50,398 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:50,398 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:50,446 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.023
2016-05-17 20:45:50,462 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.039
2016-05-17 20:45:50,462 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:50,462 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:50,463 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:50,481 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 20:45:50,490 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative
2016-05-17 20:45:50,542 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 20:45:50,542 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:50,542 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative"]
2016-05-17 20:45:50,543 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 20:45:50,543 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 20:45:50,543 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 20:45:50,544 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...leRelative/simplerelative
2016-05-17 20:45:50,546 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:50,559 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 20:45:50,563 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:50,564 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:50,564 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:50,573 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72/.tez/application_1463517950564_0001 doesn't exist and is created
2016-05-17 20:45:50,589 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:50,590 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:50,590 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72/.tez/application_1463517950564_0001
2016-05-17 20:45:50,593 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517950564_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:50,604 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:50,604 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:50,606 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:50,606 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:50,607 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72/.tez/application_1463517950564_0001/localmode-log-dir
2016-05-17 20:45:50,607 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72/.tez/application_1463517950564_0001/localmode-log-dir/history.txt.appattempt_1463517950564_0001_000000, maxErrors=10
2016-05-17 20:45:50,607 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:50,607 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517950564_0001_000000, appSubmitTime=1463517950593, launchTime=1463517950593
2016-05-17 20:45:50,610 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:50,612 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:50,616 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:50,616 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:50,616 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:49911
2016-05-17 20:45:50,618 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:50,620 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517950564_0001_000000, startTime=1463517950620
2016-05-17 20:45:50,620 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:50,620 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:50,690 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:50,690 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:50,690 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...leRelative/simplerelative, applicationId=application_1463517950564_0001, dagName=[BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72] test/(1/1) ...leRelative/simplerelative
2016-05-17 20:45:50,707 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72] test/(1/1) ...leRelative/simplerelative
2016-05-17 20:45:50,709 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517950564_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72/.tez/application_1463517950564_0001/localmode-log-dir/dag_1463517950564_0001_1.dot
2016-05-17 20:45:50,710 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72/.tez/application_1463517950564_0001/localmode-log-dir/dag_1463517950564_0001_1-tez-dag.pb.txt
2016-05-17 20:45:50,743 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517950564_0001_1
2016-05-17 20:45:50,743 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:50,743 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72] test/(1/1) ...leRelative/simplerelative
2016-05-17 20:45:50 Running Dag: dag_1463517950564_0001_1
2016-05-17 20:45:50,763 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517950564_0001_1, submitTime=1463517950708
2016-05-17 20:45:50,763 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:50,765 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FCF45131816E4FC5947C4DDFA51C1448
2016-05-17 20:45:50,765 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:50,765 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:50,766 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517950564_0001_1, initTime=1463517950764
2016-05-17 20:45:50,766 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517950564_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:50,766 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:DAG_STARTED]: dagID=dag_1463517950564_0001_1, startTime=1463517950766
2016-05-17 20:45:50,766 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:50,767 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517950564_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:50,767 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B0968C4BAD2B45BF940D458AF443ECFD : {5C0FB077FC594C67B8A6CB9357BFFD92={InputName=5C0FB077FC594C67B8A6CB9357BFFD92}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:50,767 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 5C0FB077FC594C67B8A6CB9357BFFD92, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:50,767 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD]
2016-05-17 20:45:50,767 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD]
2016-05-17 20:45:50,767 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD]
2016-05-17 20:45:50,768 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD]. Starting root input initializers: 1
2016-05-17 20:45:50,768 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:50,768 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:50,769 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448]
2016-05-17 20:45:50,772 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 20:45:50,772 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448]
2016-05-17 20:45:50,774 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 5C0FB077FC594C67B8A6CB9357BFFD92 on vertex vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD]
2016-05-17 20:45:50,774 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448]
2016-05-17 20:45:50,775 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448]
2016-05-17 20:45:50,774 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...leRelative/simplerelative, applicationId=application_1463517950564_0001, dagName=[BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72] test/(1/1) ...leRelative/simplerelative
2016-05-17 20:45:50,776 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=09D0E48849B241658D0742457F342690, vertexId=vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:50,776 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=09D0E48849B241658D0742457F342690, vertexId=vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448]
2016-05-17 20:45:50,780 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=09D0E48849B241658D0742457F342690, vertexName=FCF45131816E4FC5947C4DDFA51C1448, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:50,781 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=09D0E48849B241658D0742457F342690, vertexId=vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448]
2016-05-17 20:45:50,782 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:50,782 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 5C0FB077FC594C67B8A6CB9357BFFD92 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:50,784 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:50,784 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:50,785 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:50,784 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FCF45131816E4FC5947C4DDFA51C1448, vertexId=vertex_1463517950564_0001_1_01, initRequestedTime=1463517950769, initedTime=1463517950784, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 20:45:50,786 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448] transitioned from NEW to INITED due to event V_INIT
2016-05-17 20:45:50,788 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 20:45:50,792 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 5C0FB077FC594C67B8A6CB9357BFFD92 on vertex vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD]
2016-05-17 20:45:50,792 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:50,793 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {5C0FB077FC594C67B8A6CB9357BFFD92=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:50,793 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD] parallelism set to 1
2016-05-17 20:45:50,794 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:50,794 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517950564_0001_1_00, eventCount=1
2016-05-17 20:45:50,802 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B0968C4BAD2B45BF940D458AF443ECFD, vertexId=vertex_1463517950564_0001_1_00, initRequestedTime=1463517950767, initedTime=1463517950802, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:50,802 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:50,802 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD] with distanceFromRoot: 0
2016-05-17 20:45:50,803 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:50,803 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B0968C4BAD2B45BF940D458AF443ECFD
2016-05-17 20:45:50,803 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD]
2016-05-17 20:45:50,804 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:50,804 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517950564_0001_1_00, startRequestedTime=1463517950803, startedTime=1463517950803
2016-05-17 20:45:50,805 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:50,805 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:50,805 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463517950564_0001, with dag id: dag_1463517950564_0001_1
2016-05-17 20:45:50,810 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:TASK_STARTED]: vertexName=B0968C4BAD2B45BF940D458AF443ECFD, taskId=task_1463517950564_0001_1_00_000000, scheduledTime=1463517950805, launchTime=1463517950805
2016-05-17 20:45:50,810 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517950564_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:50,811 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463517950564_0001_1_00 for vertex: vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448] numStartedSources: 1 numSources: 1
2016-05-17 20:45:50,811 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448] with distanceFromRoot: 1
2016-05-17 20:45:50,811 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD]
2016-05-17 20:45:50,811 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B0968C4BAD2B45BF940D458AF443ECFD in vertex: FCF45131816E4FC5947C4DDFA51C1448
2016-05-17 20:45:50,812 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: FCF45131816E4FC5947C4DDFA51C1448 with 1 source tasks and 1 pending tasks
2016-05-17 20:45:50,812 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:50,812 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517950564_0001_1_01, startRequestedTime=1463517950811, startedTime=1463517950811
2016-05-17 20:45:50,812 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:50,812 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72] test/(1/1) ...leRelative/simplerelative, VertexName: B0968C4BAD2B45BF940D458AF443ECFD, VertexParallelism: 1, TaskAttemptID:attempt_1463517950564_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5C0FB077FC594C67B8A6CB9357BFFD92, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FCF45131816E4FC5947C4DDFA51C1448, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:50,813 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517950564_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:50,813 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:50,815 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:50,827 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:50,828 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:50,828 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517950564_0001_00_000001
2016-05-17 20:45:50,828 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517950564_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:50,830 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517950564_0001_00_000001
2016-05-17 20:45:50,830 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:50,831 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517950564_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:50,831 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:50,832 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517950564_0001_00_000001, launchTime=1463517950831
2016-05-17 20:45:50,833 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517950564_0001_00_000001
2016-05-17 20:45:50,834 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517950564_0001_1_00_000000_0] to container: [container_1463517950564_0001_00_000001]
2016-05-17 20:45:50,834 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517950564_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:50,834 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517950564_0001_1_00_000000_0] started. Is using containerId: [container_1463517950564_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:50,835 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B0968C4BAD2B45BF940D458AF443ECFD, taskAttemptId=attempt_1463517950564_0001_1_00_000000_0, startTime=1463517950834, containerId=container_1463517950564_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517950564_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:50,835 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517950564_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:50,835 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517950564_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:50,836 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:50,836 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517950564_0001_00_000001 given task: attempt_1463517950564_0001_1_00_000000_0
2016-05-17 20:45:50,836 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517950564_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517950564_0001_1_00_000000_0
2016-05-17 20:45:50 Starting to run new task attempt: attempt_1463517950564_0001_1_00_000000_0
2016-05-17 20:45:50,837 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517950564_0001_1_00_000000_0
2016-05-17 20:45:50,837 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:50,837 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:50,837 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:50,837 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:50,837 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:50,838 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:50,838 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72] test/(1/1) ...leRelative/simplerelative, VertexName: B0968C4BAD2B45BF940D458AF443ECFD, VertexParallelism: 1, TaskAttemptID:attempt_1463517950564_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5C0FB077FC594C67B8A6CB9357BFFD92, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FCF45131816E4FC5947C4DDFA51C1448, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:50,838 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:50,848 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517950564_0001_1_00_000000_0
2016-05-17 20:45:50,848 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:50,848 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:50,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=5C0FB077FC594C67B8A6CB9357BFFD92, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:50,851 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:50,851 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 5C0FB077FC594C67B8A6CB9357BFFD92
2016-05-17 20:45:50,852 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 5C0FB077FC594C67B8A6CB9357BFFD92
2016-05-17 20:45:50,854 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:50,856 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517950564_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:50,861 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FCF45131816E4FC5947C4DDFA51C1448, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 20:45:50,861 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:50,862 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FCF45131816E4FC5947C4DDFA51C1448
2016-05-17 20:45:50,863 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:50,865 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 20:45:50,866 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: FCF45131816E4FC5947C4DDFA51C1448
2016-05-17 20:45:50,867 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FCF45131816E4FC5947C4DDFA51C1448
2016-05-17 20:45:50,867 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:50,867 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 5C0FB077FC594C67B8A6CB9357BFFD92
2016-05-17 20:45:50,867 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 5C0FB077FC594C67B8A6CB9357BFFD92
2016-05-17 20:45:50,876 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B0968C4BAD2B45BF940D458AF443ECFD, ordinal: 0
2016-05-17 20:45:50,876 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B0968C4BAD2B45BF940D458AF443ECFD, mem on start (mb), free: 213, total: 333, max: 672
2016-05-17 20:45:50,877 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:50,877 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:50,877 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:50,877 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:50,877 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:50,877 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:50,878 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:50,878 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 20:45:50,878 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:50,879 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 20:45:50,879 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 5C0FB077FC594C67B8A6CB9357BFFD92, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:50,879 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FCF45131816E4FC5947C4DDFA51C1448, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 20:45:50,879 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:50,880 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 5C0FB077FC594C67B8A6CB9357BFFD92
2016-05-17 20:45:50,880 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:50,880 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 5C0FB077FC594C67B8A6CB9357BFFD92
2016-05-17 20:45:50,880 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 5C0FB077FC594C67B8A6CB9357BFFD92 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:50,881 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:50,881 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:50,882 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:50,882 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517950564_0001_1_00_000000_0
2016-05-17 20:45:50,886 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:50,888 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 5C0FB077FC594C67B8A6CB9357BFFD92
2016-05-17 20:45:50,889 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: 266B1E76DC3542AB9322E27C80F9F0EE
2016-05-17 20:45:50,889 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] 266B1E76DC3542AB9322E27C80F9F0EE
2016-05-17 20:45:50,893 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:50,894 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@1a2e7f83; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@62b57ee2; comparator=cascading.tuple.hadoop.util.TupleComparator@799d5a0e; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 20:45:50,894 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@409335e1
2016-05-17 20:45:50,894 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 20:45:50,895 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 20:45:50,895 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:50,896 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@409335e1
2016-05-17 20:45:50,896 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:50,896 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:50,911 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 20:45:50,912 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 20:45:50,912 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 20:45:50,912 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 20:45:50,913 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 20:45:50,914 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:50,914 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:50,914 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B0968C4BAD2B45BF940D458AF443ECFD, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:50,918 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517950564_0001_1_00_000000_0
2016-05-17 20:45:50,919 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B0968C4BAD2B45BF940D458AF443ECFD, mem on close (mb), free: 111, total: 333, max: 672
2016-05-17 20:45:50,919 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 20:45:50,920 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 20:45:50,920 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 210; bufvoid = 104857600
2016-05-17 20:45:50,921 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214360(104857440); length = 37/6553600
2016-05-17 20:45:50,926 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 20:45:50 Completed running task attempt: attempt_1463517950564_0001_1_00_000000_0
2016-05-17 20:45:50,930 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517950564_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:50,931 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517950564_0001_1_00, eventCount=1
2016-05-17 20:45:50,932 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B0968C4BAD2B45BF940D458AF443ECFD, taskAttemptId=attempt_1463517950564_0001_1_00_000000_0, startTime=1463517950834, finishTime=1463517950932, timeTaken=98, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=349700096, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517950914, Process_Duration=4, Process_End_Time=1463517950918, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 20:45:50,933 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517950564_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:50,933 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:50,933 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517950564_0001_1_00_000000_0
2016-05-17 20:45:50,934 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:TASK_FINISHED]: vertexName=B0968C4BAD2B45BF940D458AF443ECFD, taskId=task_1463517950564_0001_1_00_000000, startTime=1463517950834, finishTime=1463517950933, timeTaken=99, status=SUCCEEDED, successfulAttemptID=attempt_1463517950564_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=349700096, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517950914, Process_Duration=4, Process_End_Time=1463517950918, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 20:45:50,934 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517950564_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:50,934 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD] : 1
2016-05-17 20:45:50,934 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD]
2016-05-17 20:45:50,935 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:50,936 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:50,937 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:VERTEX_FINISHED]: vertexName=B0968C4BAD2B45BF940D458AF443ECFD, vertexId=vertex_1463517950564_0001_1_00, initRequestedTime=1463517950767, initedTime=1463517950802, startRequestedTime=1463517950803, startedTime=1463517950803, finishTime=1463517950934, timeTaken=131, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=349700096, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517950914, Process_Duration=4, Process_End_Time=1463517950918, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463517950834, firstTasksToStart=[ task_1463517950564_0001_1_00_000000 ], lastTaskFinishTime=1463517950932, lastTasksToFinish=[ task_1463517950564_0001_1_00_000000 ], minTaskDuration=98, maxTaskDuration=98, avgTaskDuration=98.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517950564_0001_1_00_000000 ], longestDurationTasks=[ task_1463517950564_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:50,938 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:50,938 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:50,944 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517950564_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:50,944 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448] attempt: attempt_1463517950564_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 20:45:50,944 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: FCF45131816E4FC5947C4DDFA51C1448
2016-05-17 20:45:50,944 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448]
2016-05-17 20:45:50,945 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517950564_0001_00_000001, stoppedTime=1463517950945, exitStatus=0
2016-05-17 20:45:50,945 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:50,944 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=349700096, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463517950914, Process_Duration=4, Process_End_Time=1463517950918, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 20:45:50,945 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517950564_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:50,948 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:50,948 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:50,948 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:50,949 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517950564_0001_00_000001
2016-05-17 20:45:50,949 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:TASK_STARTED]: vertexName=FCF45131816E4FC5947C4DDFA51C1448, taskId=task_1463517950564_0001_1_01_000000, scheduledTime=1463517950949, launchTime=1463517950949
2016-05-17 20:45:50,949 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:50,949 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517950564_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:50,950 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448]
2016-05-17 20:45:50,951 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72] test/(1/1) ...leRelative/simplerelative, VertexName: FCF45131816E4FC5947C4DDFA51C1448, VertexParallelism: 1, TaskAttemptID:attempt_1463517950564_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B0968C4BAD2B45BF940D458AF443ECFD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=09D0E48849B241658D0742457F342690, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:50,951 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517950564_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:50,951 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:50,952 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517950564_0001_00_000002
2016-05-17 20:45:50,952 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517950564_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:50,953 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463517950564_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 20:45:50,954 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517950564_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 20:45:50,955 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517950564_0001_00_000002
2016-05-17 20:45:50,955 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:50,962 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517950564_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:50,962 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517950564_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:50,963 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:50,963 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517950564_0001_00_000002, launchTime=1463517950962
2016-05-17 20:45:50,963 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517950564_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:50,964 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517950564_0001_00_000001, Exiting
2016-05-17 20:45:50,964 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517950564_0001_00_000001
2016-05-17 20:45:50,964 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517950564_0001_00_000001
2016-05-17 20:45:50,976 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517950564_0001_00_000002
2016-05-17 20:45:50,986 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517950564_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:50,987 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517950564_0001_1_01_000000_0] to container: [container_1463517950564_0001_00_000002]
2016-05-17 20:45:50,987 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517950564_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:50,987 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517950564_0001_00_000002 given task: attempt_1463517950564_0001_1_01_000000_0
2016-05-17 20:45:50,987 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517950564_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517950564_0001_1_01_000000_0
2016-05-17 20:45:50,988 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517950564_0001_1_01_000000_0] started. Is using containerId: [container_1463517950564_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 20:45:50,988 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FCF45131816E4FC5947C4DDFA51C1448, taskAttemptId=attempt_1463517950564_0001_1_01_000000_0, startTime=1463517950988, containerId=container_1463517950564_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517950564_0001_00_000002/teamcity, completedLogs=
2016-05-17 20:45:50,989 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517950564_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:50,989 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517950564_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:50 Starting to run new task attempt: attempt_1463517950564_0001_1_01_000000_0
2016-05-17 20:45:50,989 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517950564_0001_1_01_000000_0
2016-05-17 20:45:50,989 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:50,990 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:50,990 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:50,990 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:50,990 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:50,990 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:50,990 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72] test/(1/1) ...leRelative/simplerelative, VertexName: FCF45131816E4FC5947C4DDFA51C1448, VertexParallelism: 1, TaskAttemptID:attempt_1463517950564_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B0968C4BAD2B45BF940D458AF443ECFD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=09D0E48849B241658D0742457F342690, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:50,991 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:50,991 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:50,987 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517950564_0001_00_000001 completed successfully
2016-05-17 20:45:50,992 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517950564_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 20:45:50,993 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517950564_0001_1_01_000000_0
2016-05-17 20:45:50,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:50,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:51,004 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B0968C4BAD2B45BF940D458AF443ECFD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 20:45:51,005 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:51,005 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B0968C4BAD2B45BF940D458AF443ECFD
2016-05-17 20:45:51,006 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:51,006 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 20:45:51,007 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 20:45:51,007 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B0968C4BAD2B45BF940D458AF443ECFD
2016-05-17 20:45:51,007 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B0968C4BAD2B45BF940D458AF443ECFD
2016-05-17 20:45:51,009 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:51,016 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517950564_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 20:45:51,022 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=09D0E48849B241658D0742457F342690, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:51,023 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:51,026 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 09D0E48849B241658D0742457F342690
2016-05-17 20:45:51,027 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:51,027 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 09D0E48849B241658D0742457F342690
2016-05-17 20:45:51,045 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FCF45131816E4FC5947C4DDFA51C1448, ordinal: 1
2016-05-17 20:45:51,046 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FCF45131816E4FC5947C4DDFA51C1448, mem on start (mb), free: 322, total: 346, max: 672
2016-05-17 20:45:51,047 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:51,047 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:51,048 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:51,055 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:51,055 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 09D0E48849B241658D0742457F342690
2016-05-17 20:45:51,055 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:51,055 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:51,055 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 20:45:51,055 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:51,056 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:51,056 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:51,056 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 20:45:51,056 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B0968C4BAD2B45BF940D458AF443ECFD, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 20:45:51,056 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 09D0E48849B241658D0742457F342690, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:51,056 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:51,056 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B0968C4BAD2B45BF940D458AF443ECFD being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:51,057 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B0968C4BAD2B45BF940D458AF443ECFD
2016-05-17 20:45:51,057 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:51,057 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:51,058 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:51,058 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 20:45:51,060 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B0968C4BAD2B45BF940D458AF443ECFD with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 20:45:51,061 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 20:45:51,061 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 20:45:51,061 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 20:45:51,062 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B0968C4BAD2B45BF940D458AF443ECFD
2016-05-17 20:45:51,065 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:51,066 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517950564_0001_1_01_000000_0
2016-05-17 20:45:51,068 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463517950564_0001_1_00_000000_0_10034, runDuration: 64892, ]
2016-05-17 20:45:51,070 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463517950564_0001_1_00_000000_0_10034] to DISK_DIRECT, CompressedSize=204, DecompressedSize=200,EndTime=1463517951070, TimeTaken=1, Rate=0.19 MB/s
2016-05-17 20:45:51,070 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B0968C4BAD2B45BF940D458AF443ECFD
2016-05-17 20:45:51,070 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-05-17 20:45:51,077 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: 266B1E76DC3542AB9322E27C80F9F0EE
2016-05-17 20:45:51,080 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 20:45:51,080 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative"], id: 09D0E48849B241658D0742457F342690
2016-05-17 20:45:51,081 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [B0968C4BAD2B45BF940D458AF443ECFD] #6 in 13ms
2016-05-17 20:45:51,082 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative"]
2016-05-17 20:45:51,083 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B0968C4BAD2B45BF940D458AF443ECFD] #6, status:false, isInterrupted:false
2016-05-17 20:45:51,083 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] 266B1E76DC3542AB9322E27C80F9F0EE, for 1 inputs
2016-05-17 20:45:51,084 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 20:45:51,085 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 204 bytes from disk
2016-05-17 20:45:51,085 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:51,085 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 20:45:51,086 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 175 bytes
2016-05-17 20:45:51,086 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B0968C4BAD2B45BF940D458AF443ECFD
2016-05-17 20:45:51,087 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 20:45:51,088 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FCF45131816E4FC5947C4DDFA51C1448, all 1 inputs ready in: 00:00:00.003
2016-05-17 20:45:51,090 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@5f9bbec5; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@6a09a88c
2016-05-17 20:45:51,095 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:51,096 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:51,097 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517950564_0001_1_01_000000_0
2016-05-17 20:45:51,097 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517950564_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 20:45:51,099 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517950564_0001_1_01_000000_0
2016-05-17 20:45:51,099 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FCF45131816E4FC5947C4DDFA51C1448, mem on close (mb), free: 319, total: 346, max: 672
2016-05-17 20:45:51,099 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B0968C4BAD2B45BF940D458AF443ECFD
2016-05-17 20:45:51,100 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 20:45:51,101 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 20:45:51,101 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:51,101 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 20:45:51,102 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 20:45:51 Completed running task attempt: attempt_1463517950564_0001_1_01_000000_0
2016-05-17 20:45:51,106 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517950564_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:51,107 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=660, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=363331584, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=23, MERGE_PHASE_TIME=28, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463517951088, Process_Duration=10, Process_End_Time=1463517951098, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=8]]
2016-05-17 20:45:51,108 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:51,108 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FCF45131816E4FC5947C4DDFA51C1448, taskAttemptId=attempt_1463517950564_0001_1_01_000000_0, startTime=1463517950988, finishTime=1463517951107, timeTaken=119, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=660, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=363331584, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=23, MERGE_PHASE_TIME=28, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517951088, Process_Duration=10, Process_End_Time=1463517951098, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-05-17 20:45:51,109 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:51,110 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517950564_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:51,110 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:51,110 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517950564_0001_1_01_000000_0
2016-05-17 20:45:51,111 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517950564_0001_00_000002
2016-05-17 20:45:51,111 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:51,111 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:TASK_FINISHED]: vertexName=FCF45131816E4FC5947C4DDFA51C1448, taskId=task_1463517950564_0001_1_01_000000, startTime=1463517950988, finishTime=1463517951110, timeTaken=122, status=SUCCEEDED, successfulAttemptID=attempt_1463517950564_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=660, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=363331584, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=23, MERGE_PHASE_TIME=28, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517951088, Process_Duration=10, Process_End_Time=1463517951098, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-05-17 20:45:51,111 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:51,111 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517950564_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:51,112 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448] : 1
2016-05-17 20:45:51,112 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448]
2016-05-17 20:45:51,113 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:51,122 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:VERTEX_FINISHED]: vertexName=FCF45131816E4FC5947C4DDFA51C1448, vertexId=vertex_1463517950564_0001_1_01, initRequestedTime=1463517950769, initedTime=1463517950784, startRequestedTime=1463517950811, startedTime=1463517950811, finishTime=1463517951112, timeTaken=301, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=660, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=363331584, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=23, MERGE_PHASE_TIME=28, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517951088, Process_Duration=10, Process_End_Time=1463517951098, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8, vertexStats=firstTaskStartTime=1463517950988, firstTasksToStart=[ task_1463517950564_0001_1_01_000000 ], lastTaskFinishTime=1463517951107, lastTasksToFinish=[ task_1463517950564_0001_1_01_000000 ], minTaskDuration=119, maxTaskDuration=119, avgTaskDuration=119.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517950564_0001_1_01_000000 ], longestDurationTasks=[ task_1463517950564_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:51,129 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:51,133 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517950564_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:51,133 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517950564_0001_00_000002, stoppedTime=1463517951133, exitStatus=0
2016-05-17 20:45:51,133 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:51,134 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517950564_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:51,134 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:51,135 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517950564_0001_1
2016-05-17 20:45:51,136 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517950564_0001_1
2016-05-17 20:45:51,136 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 09D0E48849B241658D0742457F342690 for vertex: vertex_1463517950564_0001_1_01 [FCF45131816E4FC5947C4DDFA51C1448]
2016-05-17 20:45:51,141 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517950564_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 20:45:51,141 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517950564_0001_00_000002: 30 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:51,142 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517950564_0001_00_000002, Exiting
2016-05-17 20:45:51,142 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517950564_0001_00_000002
2016-05-17 20:45:51,142 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517950564_0001_00_000002
2016-05-17 20:45:51,146 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517950564_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:51,146 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517950564_0001_00_000002 completed successfully
2016-05-17 20:45:51,165 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463517950564_0001_1_00 [B0968C4BAD2B45BF940D458AF443ECFD]
2016-05-17 20:45:51,169 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517950564_0001_1, queueSize=0
2016-05-17 20:45:51,169 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:51,170 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517950564_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517950564_0001_1, startTime=1463517950766, finishTime=1463517951165, timeTaken=399, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2442, FILE_BYTES_WRITTEN=896, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=713031680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=18, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=23, MERGE_PHASE_TIME=28, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927035902002, Process_Duration=14, Process_End_Time=2927035902016, Read_Duration=3, Tuples_Read=20, Tuples_Written=18, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=8
2016-05-17 20:45:51,170 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517950564_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:51,171 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517950564_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:51,171 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517950564_0001_00_000002
2016-05-17 20:45:51,171 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517950564_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:51 Completed Dag: dag_1463517950564_0001_1
2016-05-17 20:45:51,172 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517950564_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:51,173 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517950564_0001_1_post
2016-05-17 20:45:51,173 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:51,173 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:51,189 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...leRelative/simplerelative, applicationId=application_1463517950564_0001
2016-05-17 20:45:51,192 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:51,192 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:51,192 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:51,192 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:51,192 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:51,193 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:51,193 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:51,195 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:51,195 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:51,196 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:51,196 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:51,196 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:51,196 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:51,204 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BF89C0FF22E340D5A112E2B740CE67D9/F8FB824471A0495481211B556FFEAD72/.tez/application_1463517950564_0001
2016-05-17 20:45:51,205 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative/_temporary
2016-05-17 20:45:51,205 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.662
2016-05-17 20:45:51,320 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative
2016-05-17 20:45:51,450 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:51,451 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:51,503 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.026
2016-05-17 20:45:51,519 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.042
2016-05-17 20:45:51,519 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:51,519 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:51,520 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:51,537 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt
2016-05-17 20:45:51,553 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult
2016-05-17 20:45:51,603 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 20:45:51,603 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 20:45:51,605 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult"]
2016-05-17 20:45:51,605 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 20:45:51,605 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 20:45:51,605 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 20:45:51,608 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...SimpleResult/simpleresult
2016-05-17 20:45:51,608 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:51,621 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 20:45:51,625 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:51,626 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:51,626 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:51,637 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF/.tez/application_1463517951626_0001 doesn't exist and is created
2016-05-17 20:45:51,654 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:51,654 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:51,655 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF/.tez/application_1463517951626_0001
2016-05-17 20:45:51,658 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517951626_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:51,669 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:51,670 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:51,671 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:51,672 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:51,672 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF/.tez/application_1463517951626_0001/localmode-log-dir
2016-05-17 20:45:51,672 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF/.tez/application_1463517951626_0001/localmode-log-dir/history.txt.appattempt_1463517951626_0001_000000, maxErrors=10
2016-05-17 20:45:51,673 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:51,673 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517951626_0001_000000, appSubmitTime=1463517951658, launchTime=1463517951658
2016-05-17 20:45:51,677 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:51,679 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:51,680 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:51,680 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:51,682 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:48409
2016-05-17 20:45:51,684 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:51,688 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517951626_0001_000000, startTime=1463517951688
2016-05-17 20:45:51,688 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:51,688 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:51,754 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:51,754 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:51,755 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...SimpleResult/simpleresult, applicationId=application_1463517951626_0001, dagName=[7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF] test/(1/1) ...SimpleResult/simpleresult
2016-05-17 20:45:51,775 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF] test/(1/1) ...SimpleResult/simpleresult
2016-05-17 20:45:51,777 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517951626_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF/.tez/application_1463517951626_0001/localmode-log-dir/dag_1463517951626_0001_1.dot
2016-05-17 20:45:51,777 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF/.tez/application_1463517951626_0001/localmode-log-dir/dag_1463517951626_0001_1-tez-dag.pb.txt
2016-05-17 20:45:51,787 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517951626_0001_1
2016-05-17 20:45:51,787 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:51,787 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF] test/(1/1) ...SimpleResult/simpleresult
2016-05-17 20:45:51 Running Dag: dag_1463517951626_0001_1
2016-05-17 20:45:51,801 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:51,801 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517951626_0001_1, submitTime=1463517951775
2016-05-17 20:45:51,803 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F32E0FE10D22481EBADD54A3C5B6E7B5
2016-05-17 20:45:51,803 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:51,803 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:51,804 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517951626_0001_1, initTime=1463517951801
2016-05-17 20:45:51,805 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517951626_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:51,805 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:DAG_STARTED]: dagID=dag_1463517951626_0001_1, startTime=1463517951805
2016-05-17 20:45:51,806 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:51,805 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:51,806 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517951626_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:51,806 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B29AD20FE1E94CDB96DBAD67628E7E85 : {27D522ED170E40519DBAB82CD01BAE88={InputName=27D522ED170E40519DBAB82CD01BAE88}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:51,806 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 27D522ED170E40519DBAB82CD01BAE88, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:51,806 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85]
2016-05-17 20:45:51,806 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85]
2016-05-17 20:45:51,806 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85]
2016-05-17 20:45:51,807 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85]. Starting root input initializers: 1
2016-05-17 20:45:51,807 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...SimpleResult/simpleresult, applicationId=application_1463517951626_0001, dagName=[7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF] test/(1/1) ...SimpleResult/simpleresult
2016-05-17 20:45:51,807 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:51,807 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5]
2016-05-17 20:45:51,808 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 27D522ED170E40519DBAB82CD01BAE88 on vertex vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85]
2016-05-17 20:45:51,810 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:51,810 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 20:45:51,810 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5]
2016-05-17 20:45:51,811 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5]
2016-05-17 20:45:51,811 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5]
2016-05-17 20:45:51,811 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=8C53548AE7C04A28A8B070108FE52298, vertexId=vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:51,812 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=8C53548AE7C04A28A8B070108FE52298, vertexId=vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5]
2016-05-17 20:45:51,810 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 27D522ED170E40519DBAB82CD01BAE88 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:51,814 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:51,814 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=8C53548AE7C04A28A8B070108FE52298, vertexName=F32E0FE10D22481EBADD54A3C5B6E7B5, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:51,814 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=8C53548AE7C04A28A8B070108FE52298, vertexId=vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5]
2016-05-17 20:45:51,815 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:51,816 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:51,817 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F32E0FE10D22481EBADD54A3C5B6E7B5, vertexId=vertex_1463517951626_0001_1_01, initRequestedTime=1463517951807, initedTime=1463517951816, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 20:45:51,818 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5] transitioned from NEW to INITED due to event V_INIT
2016-05-17 20:45:51,823 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463517951626_0001, with dag id: dag_1463517951626_0001_1
2016-05-17 20:45:51,824 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-05-17 20:45:51,831 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 27D522ED170E40519DBAB82CD01BAE88 on vertex vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85]
2016-05-17 20:45:51,832 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:51,833 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {27D522ED170E40519DBAB82CD01BAE88=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:51,834 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85] parallelism set to 1
2016-05-17 20:45:51,834 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:51,835 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517951626_0001_1_00, eventCount=1
2016-05-17 20:45:51,836 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:51,838 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B29AD20FE1E94CDB96DBAD67628E7E85, vertexId=vertex_1463517951626_0001_1_00, initRequestedTime=1463517951806, initedTime=1463517951836, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:51,838 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85] with distanceFromRoot: 0
2016-05-17 20:45:51,839 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:51,839 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B29AD20FE1E94CDB96DBAD67628E7E85
2016-05-17 20:45:51,839 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85]
2016-05-17 20:45:51,840 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:51,840 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517951626_0001_1_00, startRequestedTime=1463517951839, startedTime=1463517951839
2016-05-17 20:45:51,840 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:51,841 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:51,841 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:TASK_STARTED]: vertexName=B29AD20FE1E94CDB96DBAD67628E7E85, taskId=task_1463517951626_0001_1_00_000000, scheduledTime=1463517951840, launchTime=1463517951840
2016-05-17 20:45:51,841 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517951626_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:51,841 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463517951626_0001_1_00 for vertex: vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5] numStartedSources: 1 numSources: 1
2016-05-17 20:45:51,841 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5] with distanceFromRoot: 1
2016-05-17 20:45:51,842 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85]
2016-05-17 20:45:51,842 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B29AD20FE1E94CDB96DBAD67628E7E85 in vertex: F32E0FE10D22481EBADD54A3C5B6E7B5
2016-05-17 20:45:51,847 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F32E0FE10D22481EBADD54A3C5B6E7B5 with 1 source tasks and 1 pending tasks
2016-05-17 20:45:51,848 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:51,848 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517951626_0001_1_01, startRequestedTime=1463517951842, startedTime=1463517951842
2016-05-17 20:45:51,848 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:51,848 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF] test/(1/1) ...SimpleResult/simpleresult, VertexName: B29AD20FE1E94CDB96DBAD67628E7E85, VertexParallelism: 1, TaskAttemptID:attempt_1463517951626_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=27D522ED170E40519DBAB82CD01BAE88, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F32E0FE10D22481EBADD54A3C5B6E7B5, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:51,849 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517951626_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:51,849 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:51,850 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:51,851 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:51,851 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:51,852 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517951626_0001_00_000001
2016-05-17 20:45:51,852 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517951626_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:51,854 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517951626_0001_00_000001
2016-05-17 20:45:51,854 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:51,855 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517951626_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:51,855 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:51,856 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517951626_0001_00_000001, launchTime=1463517951855
2016-05-17 20:45:51,856 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517951626_0001_00_000001
2016-05-17 20:45:51,857 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517951626_0001_1_00_000000_0] to container: [container_1463517951626_0001_00_000001]
2016-05-17 20:45:51,858 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517951626_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:51,858 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517951626_0001_1_00_000000_0] started. Is using containerId: [container_1463517951626_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:51,859 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B29AD20FE1E94CDB96DBAD67628E7E85, taskAttemptId=attempt_1463517951626_0001_1_00_000000_0, startTime=1463517951858, containerId=container_1463517951626_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517951626_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:51,859 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:51,859 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517951626_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:51,859 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517951626_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:51,860 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517951626_0001_00_000001 given task: attempt_1463517951626_0001_1_00_000000_0
2016-05-17 20:45:51,860 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517951626_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517951626_0001_1_00_000000_0
2016-05-17 20:45:51 Starting to run new task attempt: attempt_1463517951626_0001_1_00_000000_0
2016-05-17 20:45:51,861 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517951626_0001_1_00_000000_0
2016-05-17 20:45:51,861 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:51,861 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:51,861 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:51,861 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:51,861 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:51,861 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:51,862 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF] test/(1/1) ...SimpleResult/simpleresult, VertexName: B29AD20FE1E94CDB96DBAD67628E7E85, VertexParallelism: 1, TaskAttemptID:attempt_1463517951626_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=27D522ED170E40519DBAB82CD01BAE88, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F32E0FE10D22481EBADD54A3C5B6E7B5, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:51,862 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:51,864 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517951626_0001_1_00_000000_0
2016-05-17 20:45:51,864 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:51,865 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:51,866 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=27D522ED170E40519DBAB82CD01BAE88, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:51,867 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:51,867 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 27D522ED170E40519DBAB82CD01BAE88
2016-05-17 20:45:51,868 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 27D522ED170E40519DBAB82CD01BAE88
2016-05-17 20:45:51,874 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:51,874 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F32E0FE10D22481EBADD54A3C5B6E7B5, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 20:45:51,875 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:51,875 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F32E0FE10D22481EBADD54A3C5B6E7B5
2016-05-17 20:45:51,876 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:51,876 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 27D522ED170E40519DBAB82CD01BAE88
2016-05-17 20:45:51,877 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 27D522ED170E40519DBAB82CD01BAE88
2016-05-17 20:45:51,877 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:51,878 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 20:45:51,878 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F32E0FE10D22481EBADD54A3C5B6E7B5
2016-05-17 20:45:51,881 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517951626_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:51,881 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F32E0FE10D22481EBADD54A3C5B6E7B5
2016-05-17 20:45:51,890 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B29AD20FE1E94CDB96DBAD67628E7E85, ordinal: 0
2016-05-17 20:45:51,890 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B29AD20FE1E94CDB96DBAD67628E7E85, mem on start (mb), free: 225, total: 346, max: 672
2016-05-17 20:45:51,890 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:51,891 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:51,891 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:51,891 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:51,892 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:51,892 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:51,892 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:51,893 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 20:45:51,893 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:51,894 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 20:45:51,894 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 27D522ED170E40519DBAB82CD01BAE88, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:51,894 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F32E0FE10D22481EBADD54A3C5B6E7B5, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 20:45:51,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:51,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 27D522ED170E40519DBAB82CD01BAE88
2016-05-17 20:45:51,895 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:51,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 27D522ED170E40519DBAB82CD01BAE88
2016-05-17 20:45:51,896 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 27D522ED170E40519DBAB82CD01BAE88 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:51,896 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:51,897 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:51,897 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:51,898 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517951626_0001_1_00_000000_0
2016-05-17 20:45:51,901 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 20:45:51,902 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:51,903 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@42e0615c
2016-05-17 20:45:51,904 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"] streamed: true, id: 27D522ED170E40519DBAB82CD01BAE88
2016-05-17 20:45:51,904 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: A64C9A1178744E92B0B90EA7911B1CFF
2016-05-17 20:45:51,904 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] A64C9A1178744E92B0B90EA7911B1CFF
2016-05-17 20:45:51,905 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@42e0615c
2016-05-17 20:45:51,912 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:51,912 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:51,915 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@11c76237; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@390111ff; comparator=cascading.tuple.hadoop.util.TupleComparator@75449995; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 20:45:51,916 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 20:45:51,916 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 20:45:51,917 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:51,930 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 20:45:51,931 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 20:45:51,931 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 20:45:51,931 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 20:45:51,932 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 20:45:52,015 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 20:45:52,016 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 20:45:52,301 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 20:45:52,301 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:52,302 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B29AD20FE1E94CDB96DBAD67628E7E85, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:52,306 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517951626_0001_1_00_000000_0
2016-05-17 20:45:52,306 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B29AD20FE1E94CDB96DBAD67628E7E85, mem on close (mb), free: 416, total: 541, max: 672
2016-05-17 20:45:52,307 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 20:45:52,307 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 20:45:52,307 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 22; bufvoid = 104857600
2016-05-17 20:45:52,308 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214392(104857568); length = 5/6553600
2016-05-17 20:45:52,311 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 20:45:52 Completed running task attempt: attempt_1463517951626_0001_1_00_000000_0
2016-05-17 20:45:52,317 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517951626_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:52,318 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517951626_0001_1_00, eventCount=1
2016-05-17 20:45:52,318 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:52,318 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B29AD20FE1E94CDB96DBAD67628E7E85, taskAttemptId=attempt_1463517951626_0001_1_00_000000_0, startTime=1463517951858, finishTime=1463517952318, timeTaken=460, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=68, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=2, GC_TIME_MILLIS=266, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567279616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517952302, Process_Duration=4, Process_End_Time=1463517952306, Read_Duration=2, Tuples_Read=5, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 20:45:52,319 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517951626_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:52,319 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:52,319 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517951626_0001_1_00_000000_0
2016-05-17 20:45:52,320 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:TASK_FINISHED]: vertexName=B29AD20FE1E94CDB96DBAD67628E7E85, taskId=task_1463517951626_0001_1_00_000000, startTime=1463517951858, finishTime=1463517952319, timeTaken=461, status=SUCCEEDED, successfulAttemptID=attempt_1463517951626_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=68, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=2, GC_TIME_MILLIS=266, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567279616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517952302, Process_Duration=4, Process_End_Time=1463517952306, Read_Duration=2, Tuples_Read=5, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 20:45:52,320 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517951626_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:52,320 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85] : 1
2016-05-17 20:45:52,320 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85]
2016-05-17 20:45:52,321 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:52,323 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:52,322 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:VERTEX_FINISHED]: vertexName=B29AD20FE1E94CDB96DBAD67628E7E85, vertexId=vertex_1463517951626_0001_1_00, initRequestedTime=1463517951806, initedTime=1463517951836, startRequestedTime=1463517951839, startedTime=1463517951839, finishTime=1463517952321, timeTaken=482, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=68, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=2, GC_TIME_MILLIS=266, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567279616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517952302, Process_Duration=4, Process_End_Time=1463517952306, Read_Duration=2, Tuples_Read=5, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463517951858, firstTasksToStart=[ task_1463517951626_0001_1_00_000000 ], lastTaskFinishTime=1463517952318, lastTasksToFinish=[ task_1463517951626_0001_1_00_000000 ], minTaskDuration=460, maxTaskDuration=460, avgTaskDuration=460.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517951626_0001_1_00_000000 ], longestDurationTasks=[ task_1463517951626_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:52,324 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:52,325 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517951626_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:52,325 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5] attempt: attempt_1463517951626_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 20:45:52,326 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F32E0FE10D22481EBADD54A3C5B6E7B5
2016-05-17 20:45:52,326 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5]
2016-05-17 20:45:52,326 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517951626_0001_00_000001, stoppedTime=1463517952326, exitStatus=0
2016-05-17 20:45:52,327 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517951626_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:52,327 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:52,326 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:52,328 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:52,329 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:TASK_STARTED]: vertexName=F32E0FE10D22481EBADD54A3C5B6E7B5, taskId=task_1463517951626_0001_1_01_000000, scheduledTime=1463517952328, launchTime=1463517952328
2016-05-17 20:45:52,329 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517951626_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:52,329 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5]
2016-05-17 20:45:52,330 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463517951626_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 20:45:52,330 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF] test/(1/1) ...SimpleResult/simpleresult, VertexName: F32E0FE10D22481EBADD54A3C5B6E7B5, VertexParallelism: 1, TaskAttemptID:attempt_1463517951626_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B29AD20FE1E94CDB96DBAD67628E7E85, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=8C53548AE7C04A28A8B070108FE52298, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:52,331 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517951626_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:52,331 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517951626_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 20:45:52,331 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=68, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=2, GC_TIME_MILLIS=266, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567279616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463517952302, Process_Duration=4, Process_End_Time=1463517952306, Read_Duration=2, Tuples_Read=5, Tuples_Written=2, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-17 20:45:52,332 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:52,332 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:52,332 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:52,335 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517951626_0001_00_000001
2016-05-17 20:45:52,336 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517951626_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:52,336 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517951626_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:52,335 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517951626_0001_00_000002
2016-05-17 20:45:52,337 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517951626_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:52,338 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517951626_0001_00_000001, Exiting
2016-05-17 20:45:52,338 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517951626_0001_00_000001
2016-05-17 20:45:52,338 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517951626_0001_00_000001
2016-05-17 20:45:52,339 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517951626_0001_00_000002
2016-05-17 20:45:52,341 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 20:45:52,342 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517951626_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:52,343 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517951626_0001_00_000001 completed successfully
2016-05-17 20:45:52,344 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517951626_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 20:45:52,345 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517951626_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:52,346 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:52,348 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517951626_0001_00_000002, launchTime=1463517952346
2016-05-17 20:45:52,349 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517951626_0001_00_000002
2016-05-17 20:45:52,350 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517951626_0001_1_01_000000_0] to container: [container_1463517951626_0001_00_000002]
2016-05-17 20:45:52,350 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517951626_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:52,351 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517951626_0001_1_01_000000_0] started. Is using containerId: [container_1463517951626_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 20:45:52,352 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F32E0FE10D22481EBADD54A3C5B6E7B5, taskAttemptId=attempt_1463517951626_0001_1_01_000000_0, startTime=1463517952351, containerId=container_1463517951626_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517951626_0001_00_000002/teamcity, completedLogs=
2016-05-17 20:45:52,352 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517951626_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:52,352 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517951626_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:52,352 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:52,354 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517951626_0001_00_000002 given task: attempt_1463517951626_0001_1_01_000000_0
2016-05-17 20:45:52,355 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517951626_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517951626_0001_1_01_000000_0
2016-05-17 20:45:52 Starting to run new task attempt: attempt_1463517951626_0001_1_01_000000_0
2016-05-17 20:45:52,355 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517951626_0001_1_01_000000_0
2016-05-17 20:45:52,356 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:52,356 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:52,356 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:52,356 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:52,356 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:52,356 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:52,356 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF] test/(1/1) ...SimpleResult/simpleresult, VertexName: F32E0FE10D22481EBADD54A3C5B6E7B5, VertexParallelism: 1, TaskAttemptID:attempt_1463517951626_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B29AD20FE1E94CDB96DBAD67628E7E85, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=8C53548AE7C04A28A8B070108FE52298, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:52,357 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:52,361 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517951626_0001_1_01_000000_0
2016-05-17 20:45:52,362 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:52,363 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:52,364 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517951626_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 20:45:52,365 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B29AD20FE1E94CDB96DBAD67628E7E85, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 20:45:52,366 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:52,367 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B29AD20FE1E94CDB96DBAD67628E7E85
2016-05-17 20:45:52,367 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=8C53548AE7C04A28A8B070108FE52298, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:52,367 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:52,369 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:52,370 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 8C53548AE7C04A28A8B070108FE52298
2016-05-17 20:45:52,370 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:52,370 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 8C53548AE7C04A28A8B070108FE52298
2016-05-17 20:45:52,371 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:52,371 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 20:45:52,372 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 20:45:52,374 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B29AD20FE1E94CDB96DBAD67628E7E85
2016-05-17 20:45:52,375 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B29AD20FE1E94CDB96DBAD67628E7E85
2016-05-17 20:45:52,388 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:52,388 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 8C53548AE7C04A28A8B070108FE52298
2016-05-17 20:45:52,388 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F32E0FE10D22481EBADD54A3C5B6E7B5, ordinal: 1
2016-05-17 20:45:52,388 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F32E0FE10D22481EBADD54A3C5B6E7B5, mem on start (mb), free: 409, total: 541, max: 672
2016-05-17 20:45:52,388 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:52,388 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:52,388 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:52,388 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:52,389 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:52,389 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:52,389 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:52,389 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 20:45:52,389 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:52,389 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 20:45:52,390 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 8C53548AE7C04A28A8B070108FE52298, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:52,390 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B29AD20FE1E94CDB96DBAD67628E7E85, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 20:45:52,390 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:52,390 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B29AD20FE1E94CDB96DBAD67628E7E85
2016-05-17 20:45:52,390 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B29AD20FE1E94CDB96DBAD67628E7E85 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:52,391 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:52,393 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:52,393 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 20:45:52,393 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:52,393 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B29AD20FE1E94CDB96DBAD67628E7E85 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 20:45:52,393 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 20:45:52,394 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 20:45:52,394 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 20:45:52,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B29AD20FE1E94CDB96DBAD67628E7E85
2016-05-17 20:45:52,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:52,396 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517951626_0001_1_01_000000_0
2016-05-17 20:45:52,398 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463517951626_0001_1_00_000000_0_10040, runDuration: 439431, ]
2016-05-17 20:45:52,399 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-05-17 20:45:52,400 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: A64C9A1178744E92B0B90EA7911B1CFF
2016-05-17 20:45:52,400 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult"], id: 8C53548AE7C04A28A8B070108FE52298
2016-05-17 20:45:52,401 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult"]
2016-05-17 20:45:52,401 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] A64C9A1178744E92B0B90EA7911B1CFF, for 1 inputs
2016-05-17 20:45:52,401 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463517951626_0001_1_00_000000_0_10040] to DISK_DIRECT, CompressedSize=36, DecompressedSize=32,EndTime=1463517952401, TimeTaken=2, Rate=0.02 MB/s
2016-05-17 20:45:52,402 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B29AD20FE1E94CDB96DBAD67628E7E85
2016-05-17 20:45:52,402 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 20:45:52,402 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [B29AD20FE1E94CDB96DBAD67628E7E85] #7 in 3ms
2016-05-17 20:45:52,402 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B29AD20FE1E94CDB96DBAD67628E7E85] #7, status:false, isInterrupted:false
2016-05-17 20:45:52,403 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 20:45:52,404 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 36 bytes from disk
2016-05-17 20:45:52,404 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:52,404 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 20:45:52,404 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 17 bytes
2016-05-17 20:45:52,404 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B29AD20FE1E94CDB96DBAD67628E7E85
2016-05-17 20:45:52,405 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F32E0FE10D22481EBADD54A3C5B6E7B5, all 1 inputs ready in: 00:00:00.003
2016-05-17 20:45:52,405 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 20:45:52,407 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@4ce1ffdd; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@9652d9a
2016-05-17 20:45:52,408 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:52,411 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:52,412 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517951626_0001_1_01_000000_0
2016-05-17 20:45:52,412 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517951626_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 20:45:52,414 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517951626_0001_1_01_000000_0
2016-05-17 20:45:52,414 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F32E0FE10D22481EBADD54A3C5B6E7B5, mem on close (mb), free: 407, total: 541, max: 672
2016-05-17 20:45:52,414 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B29AD20FE1E94CDB96DBAD67628E7E85
2016-05-17 20:45:52,414 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 20:45:52,415 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 20:45:52,415 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:52,416 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 20:45:52,416 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 20:45:52 Completed running task attempt: attempt_1463517951626_0001_1_01_000000_0
2016-05-17 20:45:52,417 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517951626_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:52,419 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F32E0FE10D22481EBADD54A3C5B6E7B5, taskAttemptId=attempt_1463517951626_0001_1_01_000000_0, startTime=1463517952351, finishTime=1463517952418, timeTaken=67, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=68, FILE_BYTES_WRITTEN=24, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567279616, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517952405, Process_Duration=9, Process_End_Time=1463517952414, Read_Duration=0, Tuples_Read=2, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=2
2016-05-17 20:45:52,419 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517951626_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:52,419 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:52,419 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517951626_0001_1_01_000000_0
2016-05-17 20:45:52,419 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:52,420 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:TASK_FINISHED]: vertexName=F32E0FE10D22481EBADD54A3C5B6E7B5, taskId=task_1463517951626_0001_1_01_000000, startTime=1463517952351, finishTime=1463517952419, timeTaken=68, status=SUCCEEDED, successfulAttemptID=attempt_1463517951626_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=68, FILE_BYTES_WRITTEN=24, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567279616, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517952405, Process_Duration=9, Process_End_Time=1463517952414, Read_Duration=0, Tuples_Read=2, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=2
2016-05-17 20:45:52,420 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517951626_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:52,420 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5] : 1
2016-05-17 20:45:52,421 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5]
2016-05-17 20:45:52,420 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:52,422 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:52,422 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:VERTEX_FINISHED]: vertexName=F32E0FE10D22481EBADD54A3C5B6E7B5, vertexId=vertex_1463517951626_0001_1_01, initRequestedTime=1463517951807, initedTime=1463517951816, startRequestedTime=1463517951842, startedTime=1463517951842, finishTime=1463517952421, timeTaken=579, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=68, FILE_BYTES_WRITTEN=24, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567279616, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517952405, Process_Duration=9, Process_End_Time=1463517952414, Read_Duration=0, Tuples_Read=2, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=2, vertexStats=firstTaskStartTime=1463517952351, firstTasksToStart=[ task_1463517951626_0001_1_01_000000 ], lastTaskFinishTime=1463517952418, lastTasksToFinish=[ task_1463517951626_0001_1_01_000000 ], minTaskDuration=67, maxTaskDuration=67, avgTaskDuration=67.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517951626_0001_1_01_000000 ], longestDurationTasks=[ task_1463517951626_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:52,422 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:52,423 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=68, FILE_BYTES_WRITTEN=24, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567279616, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463517952405, Process_Duration=9, Process_End_Time=1463517952414, Read_Duration=0, Tuples_Read=2, Tuples_Written=2, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=2]]
2016-05-17 20:45:52,423 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:52,423 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:52,423 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517951626_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:52,424 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517951626_0001_00_000002, stoppedTime=1463517952424, exitStatus=0
2016-05-17 20:45:52,425 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517951626_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:52,425 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:52,426 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517951626_0001_1
2016-05-17 20:45:52,424 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517951626_0001_00_000002
2016-05-17 20:45:52,427 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517951626_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 20:45:52,427 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517951626_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:52,427 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517951626_0001_00_000002, Exiting
2016-05-17 20:45:52,427 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517951626_0001_00_000002
2016-05-17 20:45:52,428 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517951626_0001_00_000002
2016-05-17 20:45:52,427 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517951626_0001_1
2016-05-17 20:45:52,424 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:52,429 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 8C53548AE7C04A28A8B070108FE52298 for vertex: vertex_1463517951626_0001_1_01 [F32E0FE10D22481EBADD54A3C5B6E7B5]
2016-05-17 20:45:52,429 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517951626_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:52,429 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517951626_0001_00_000002 completed successfully
2016-05-17 20:45:52,439 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463517951626_0001_1_00 [B29AD20FE1E94CDB96DBAD67628E7E85]
2016-05-17 20:45:52,442 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517951626_0001_1, queueSize=0
2016-05-17 20:45:52,442 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:52,443 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517951626_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517951626_0001_1, startTime=1463517951805, finishTime=1463517952439, timeTaken=634, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=130, FILE_BYTES_WRITTEN=92, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=4, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=266, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1134559232, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=4, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927035904707, Process_Duration=13, Process_End_Time=2927035904720, Read_Duration=2, Tuples_Read=7, Tuples_Written=4, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=2
2016-05-17 20:45:52,443 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517951626_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:52,443 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517951626_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:52,444 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517951626_0001_00_000002
2016-05-17 20:45:52,444 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517951626_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:52 Completed Dag: dag_1463517951626_0001_1
2016-05-17 20:45:52,446 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517951626_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:52,446 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517951626_0001_1_post
2016-05-17 20:45:52,447 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:52,447 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:52,452 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...SimpleResult/simpleresult, applicationId=application_1463517951626_0001
2016-05-17 20:45:52,455 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:52,455 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:52,456 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:52,456 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:52,457 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:52,458 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:52,458 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:52,460 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:52,461 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:52,461 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:52,461 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:52,464 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:52,464 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:52,475 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7FC9BE5A95A24C54A0CBB43E74D437B6/DBC71F03B4104312A51FB050B31892CF/.tez/application_1463517951626_0001
2016-05-17 20:45:52,476 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult/_temporary
2016-05-17 20:45:52,477 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.872
2016-05-17 20:45:52,515 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult
2016-05-17 20:45:52,541 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:52,541 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:52,577 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-17 20:45:52,585 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-05-17 20:45:52,585 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 20:45:52,586 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 20:45:52,586 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:52,607 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 20:45:52,608 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall
2016-05-17 20:45:52,621 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 20:45:52,621 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:52,621 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall"]
2016-05-17 20:45:52,622 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 20:45:52,622 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 20:45:52,622 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 20:45:52,624 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...m/testFilterAll/filterall
2016-05-17 20:45:52,625 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:52,637 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 20:45:52,641 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:52,642 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:52,642 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:52,651 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/698E15FA8FE04110888027F1EDE9B0F8/ADC63FB509664D6C9EDAB5343D3E587C/.tez/application_1463517952642_0001 doesn't exist and is created
2016-05-17 20:45:52,668 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:52,668 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:52,669 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/698E15FA8FE04110888027F1EDE9B0F8/ADC63FB509664D6C9EDAB5343D3E587C/.tez/application_1463517952642_0001
2016-05-17 20:45:52,673 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517952642_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:52,683 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:52,685 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:52,687 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:52,688 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:52,689 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/698E15FA8FE04110888027F1EDE9B0F8/ADC63FB509664D6C9EDAB5343D3E587C/.tez/application_1463517952642_0001/localmode-log-dir
2016-05-17 20:45:52,689 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/698E15FA8FE04110888027F1EDE9B0F8/ADC63FB509664D6C9EDAB5343D3E587C/.tez/application_1463517952642_0001/localmode-log-dir/history.txt.appattempt_1463517952642_0001_000000, maxErrors=10
2016-05-17 20:45:52,689 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:52,690 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517952642_0001_000000, appSubmitTime=1463517952672, launchTime=1463517952672
2016-05-17 20:45:52,698 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:52,698 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:46736
2016-05-17 20:45:52,700 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:52,702 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:52,702 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:52,703 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:52,705 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517952642_0001_000000, startTime=1463517952705
2016-05-17 20:45:52,706 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:52,706 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:52,768 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:52,768 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:52,768 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...m/testFilterAll/filterall, applicationId=application_1463517952642_0001, dagName=[698E15FA8FE04110888027F1EDE9B0F8/ADC63FB509664D6C9EDAB5343D3E587C] test/(1/1) ...m/testFilterAll/filterall
2016-05-17 20:45:52,784 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [698E15FA8FE04110888027F1EDE9B0F8/ADC63FB509664D6C9EDAB5343D3E587C] test/(1/1) ...m/testFilterAll/filterall
2016-05-17 20:45:52,786 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517952642_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/698E15FA8FE04110888027F1EDE9B0F8/ADC63FB509664D6C9EDAB5343D3E587C/.tez/application_1463517952642_0001/localmode-log-dir/dag_1463517952642_0001_1.dot
2016-05-17 20:45:52,787 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/698E15FA8FE04110888027F1EDE9B0F8/ADC63FB509664D6C9EDAB5343D3E587C/.tez/application_1463517952642_0001/localmode-log-dir/dag_1463517952642_0001_1-tez-dag.pb.txt
2016-05-17 20:45:52,795 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517952642_0001_1
2016-05-17 20:45:52,795 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:52,795 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [698E15FA8FE04110888027F1EDE9B0F8/ADC63FB509664D6C9EDAB5343D3E587C] test/(1/1) ...m/testFilterAll/filterall
2016-05-17 20:45:52 Running Dag: dag_1463517952642_0001_1
2016-05-17 20:45:52,807 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517952642_0001_1, submitTime=1463517952785
2016-05-17 20:45:52,807 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:52,808 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FC38A02B300045A5AACF847FDC408ADB
2016-05-17 20:45:52,808 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:52,808 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517952642_0001_1, initTime=1463517952807
2016-05-17 20:45:52,808 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:52,809 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517952642_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:52,809 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:DAG_STARTED]: dagID=dag_1463517952642_0001_1, startTime=1463517952809
2016-05-17 20:45:52,810 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:52,810 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517952642_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:52,810 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:52,810 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FC38A02B300045A5AACF847FDC408ADB : {30973D9C33E44B84B0BC1B2C410B42D3={InputName=30973D9C33E44B84B0BC1B2C410B42D3}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:52,810 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 30973D9C33E44B84B0BC1B2C410B42D3, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:52,810 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB]
2016-05-17 20:45:52,811 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB]
2016-05-17 20:45:52,811 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB]
2016-05-17 20:45:52,812 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB]. Starting root input initializers: 1
2016-05-17 20:45:52,812 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:52,813 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 30973D9C33E44B84B0BC1B2C410B42D3 on vertex vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB]
2016-05-17 20:45:52,814 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:52,815 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 30973D9C33E44B84B0BC1B2C410B42D3 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:52,815 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:52,815 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:52,815 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...m/testFilterAll/filterall, applicationId=application_1463517952642_0001, dagName=[698E15FA8FE04110888027F1EDE9B0F8/ADC63FB509664D6C9EDAB5343D3E587C] test/(1/1) ...m/testFilterAll/filterall
2016-05-17 20:45:52,817 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 20:45:52,818 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 30973D9C33E44B84B0BC1B2C410B42D3 on vertex vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB]
2016-05-17 20:45:52,818 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:52,818 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {30973D9C33E44B84B0BC1B2C410B42D3=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:52,819 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB] parallelism set to 1
2016-05-17 20:45:52,819 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:52,820 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517952642_0001_1_00, eventCount=1
2016-05-17 20:45:52,820 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB]
2016-05-17 20:45:52,820 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=6515355947F84C98A03A787DA01532ED, vertexId=vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:52,820 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=6515355947F84C98A03A787DA01532ED, vertexId=vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB]
2016-05-17 20:45:52,822 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=6515355947F84C98A03A787DA01532ED, vertexName=FC38A02B300045A5AACF847FDC408ADB, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:52,822 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=6515355947F84C98A03A787DA01532ED, vertexId=vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB]
2016-05-17 20:45:52,823 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:52,823 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FC38A02B300045A5AACF847FDC408ADB, vertexId=vertex_1463517952642_0001_1_00, initRequestedTime=1463517952810, initedTime=1463517952823, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:52,824 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB] with distanceFromRoot: 0
2016-05-17 20:45:52,824 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:52,824 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FC38A02B300045A5AACF847FDC408ADB
2016-05-17 20:45:52,824 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB]
2016-05-17 20:45:52,825 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:52,825 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517952642_0001_1_00, startRequestedTime=1463517952824, startedTime=1463517952824
2016-05-17 20:45:52,825 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:52,826 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:52,826 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:TASK_STARTED]: vertexName=FC38A02B300045A5AACF847FDC408ADB, taskId=task_1463517952642_0001_1_00_000000, scheduledTime=1463517952825, launchTime=1463517952825
2016-05-17 20:45:52,826 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517952642_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:52,826 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB]
2016-05-17 20:45:52,827 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [698E15FA8FE04110888027F1EDE9B0F8/ADC63FB509664D6C9EDAB5343D3E587C] test/(1/1) ...m/testFilterAll/filterall, VertexName: FC38A02B300045A5AACF847FDC408ADB, VertexParallelism: 1, TaskAttemptID:attempt_1463517952642_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=30973D9C33E44B84B0BC1B2C410B42D3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=6515355947F84C98A03A787DA01532ED, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:52,827 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517952642_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:52,827 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:52,828 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:52,829 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:52,829 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:52,829 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517952642_0001_00_000001
2016-05-17 20:45:52,829 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517952642_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:52,831 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517952642_0001_00_000001
2016-05-17 20:45:52,831 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:52,831 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463517952642_0001, with dag id: dag_1463517952642_0001_1
2016-05-17 20:45:52,832 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517952642_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:52,833 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:52,833 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517952642_0001_00_000001, launchTime=1463517952833
2016-05-17 20:45:52,834 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517952642_0001_00_000001
2016-05-17 20:45:52,835 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517952642_0001_1_00_000000_0] to container: [container_1463517952642_0001_00_000001]
2016-05-17 20:45:52,835 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517952642_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:52,836 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517952642_0001_00_000001 given task: attempt_1463517952642_0001_1_00_000000_0
2016-05-17 20:45:52,836 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517952642_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517952642_0001_1_00_000000_0
2016-05-17 20:45:52,836 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517952642_0001_1_00_000000_0] started. Is using containerId: [container_1463517952642_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:52,837 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:52 Starting to run new task attempt: attempt_1463517952642_0001_1_00_000000_0
2016-05-17 20:45:52,837 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FC38A02B300045A5AACF847FDC408ADB, taskAttemptId=attempt_1463517952642_0001_1_00_000000_0, startTime=1463517952836, containerId=container_1463517952642_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517952642_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:52,837 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517952642_0001_1_00_000000_0
2016-05-17 20:45:52,837 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:52,838 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:52,838 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:52,837 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517952642_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:52,838 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517952642_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:52,838 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:52,839 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:52,839 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:52,839 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [698E15FA8FE04110888027F1EDE9B0F8/ADC63FB509664D6C9EDAB5343D3E587C] test/(1/1) ...m/testFilterAll/filterall, VertexName: FC38A02B300045A5AACF847FDC408ADB, VertexParallelism: 1, TaskAttemptID:attempt_1463517952642_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=30973D9C33E44B84B0BC1B2C410B42D3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=6515355947F84C98A03A787DA01532ED, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:52,839 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:52,840 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517952642_0001_1_00_000000_0
2016-05-17 20:45:52,840 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:52,840 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:52,840 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517952642_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:52,841 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=30973D9C33E44B84B0BC1B2C410B42D3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:52,842 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=6515355947F84C98A03A787DA01532ED, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:52,841 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:52,842 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:52,842 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 30973D9C33E44B84B0BC1B2C410B42D3
2016-05-17 20:45:52,843 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 30973D9C33E44B84B0BC1B2C410B42D3
2016-05-17 20:45:52,844 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:52,845 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:52,845 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 30973D9C33E44B84B0BC1B2C410B42D3
2016-05-17 20:45:52,845 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 30973D9C33E44B84B0BC1B2C410B42D3
2016-05-17 20:45:52,842 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:52,847 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 6515355947F84C98A03A787DA01532ED
2016-05-17 20:45:52,847 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:52,848 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 6515355947F84C98A03A787DA01532ED
2016-05-17 20:45:52,855 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FC38A02B300045A5AACF847FDC408ADB, ordinal: 0
2016-05-17 20:45:52,855 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FC38A02B300045A5AACF847FDC408ADB, mem on start (mb), free: 342, total: 541, max: 672
2016-05-17 20:45:52,855 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:52,856 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:52,856 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:52,860 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:52,860 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 6515355947F84C98A03A787DA01532ED
2016-05-17 20:45:52,860 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:52,861 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:52,861 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:52,861 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:52,861 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:52,861 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:52,861 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:52,861 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 20:45:52,861 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 30973D9C33E44B84B0BC1B2C410B42D3, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:52,862 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 6515355947F84C98A03A787DA01532ED, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:52,862 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:52,862 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 30973D9C33E44B84B0BC1B2C410B42D3 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:52,862 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 30973D9C33E44B84B0BC1B2C410B42D3
2016-05-17 20:45:52,862 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:52,863 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 30973D9C33E44B84B0BC1B2C410B42D3
2016-05-17 20:45:52,862 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:52,863 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:52,863 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:52,863 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517952642_0001_1_00_000000_0
2016-05-17 20:45:52,864 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:52,865 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@186f5f95
2016-05-17 20:45:52,866 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@186f5f95
2016-05-17 20:45:52,866 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:52,866 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:52,866 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:52,867 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 30973D9C33E44B84B0BC1B2C410B42D3
2016-05-17 20:45:52,867 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall"], id: 6515355947F84C98A03A787DA01532ED
2016-05-17 20:45:52,867 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall"]
2016-05-17 20:45:52,868 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:52,868 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:52,868 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FC38A02B300045A5AACF847FDC408ADB, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:52,870 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:52,870 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:52,870 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517952642_0001_1_00_000000_0
2016-05-17 20:45:52,870 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517952642_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 20:45:52,871 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517952642_0001_1_00_000000_0
2016-05-17 20:45:52,871 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FC38A02B300045A5AACF847FDC408ADB, mem on close (mb), free: 340, total: 541, max: 672
2016-05-17 20:45:52 Completed running task attempt: attempt_1463517952642_0001_1_00_000000_0
2016-05-17 20:45:52,872 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517952642_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:52,873 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 18 [[File System Counters FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567279616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463517952868, Process_Duration=3, Process_End_Time=1463517952871, Read_Duration=1, Tuples_Read=10][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 20:45:52,873 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:52,873 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FC38A02B300045A5AACF847FDC408ADB, taskAttemptId=attempt_1463517952642_0001_1_00_000000_0, startTime=1463517952836, finishTime=1463517952873, timeTaken=37, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567279616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517952868, Process_Duration=3, Process_End_Time=1463517952871, Read_Duration=1, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 20:45:52,873 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:52,874 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517952642_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:52,874 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:52,874 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:52,874 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517952642_0001_1_00_000000_0
2016-05-17 20:45:52,874 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517952642_0001_00_000001
2016-05-17 20:45:52,875 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:TASK_FINISHED]: vertexName=FC38A02B300045A5AACF847FDC408ADB, taskId=task_1463517952642_0001_1_00_000000, startTime=1463517952836, finishTime=1463517952874, timeTaken=38, status=SUCCEEDED, successfulAttemptID=attempt_1463517952642_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567279616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517952868, Process_Duration=3, Process_End_Time=1463517952871, Read_Duration=1, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 20:45:52,875 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517952642_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:52,875 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:52,876 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB] : 1
2016-05-17 20:45:52,876 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB]
2016-05-17 20:45:52,877 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:52,877 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:VERTEX_FINISHED]: vertexName=FC38A02B300045A5AACF847FDC408ADB, vertexId=vertex_1463517952642_0001_1_00, initRequestedTime=1463517952810, initedTime=1463517952823, startRequestedTime=1463517952824, startedTime=1463517952824, finishTime=1463517952876, timeTaken=52, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567279616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517952868, Process_Duration=3, Process_End_Time=1463517952871, Read_Duration=1, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463517952836, firstTasksToStart=[ task_1463517952642_0001_1_00_000000 ], lastTaskFinishTime=1463517952873, lastTasksToFinish=[ task_1463517952642_0001_1_00_000000 ], minTaskDuration=37, maxTaskDuration=37, avgTaskDuration=37.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517952642_0001_1_00_000000 ], longestDurationTasks=[ task_1463517952642_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:52,877 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:52,877 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517952642_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:52,878 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517952642_0001_00_000001, stoppedTime=1463517952878, exitStatus=0
2016-05-17 20:45:52,878 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517952642_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:52,878 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:52,878 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 20:45:52,878 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517952642_0001_1
2016-05-17 20:45:52,878 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517952642_0001_1
2016-05-17 20:45:52,878 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 6515355947F84C98A03A787DA01532ED for vertex: vertex_1463517952642_0001_1_00 [FC38A02B300045A5AACF847FDC408ADB]
2016-05-17 20:45:52,885 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517952642_0001_1, queueSize=0
2016-05-17 20:45:52,886 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:52,886 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517952642_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517952642_0001_1, startTime=1463517952809, finishTime=1463517952885, timeTaken=76, status=SUCCEEDED, diagnostics=, counters=Counters: 21, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567279616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517952868, Process_Duration=3, Process_End_Time=1463517952871, Read_Duration=1, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 20:45:52,886 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517952642_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:52,886 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517952642_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:52,886 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463517952642_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 20:45:52 Completed Dag: dag_1463517952642_0001_1
2016-05-17 20:45:52,887 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517952642_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:52,887 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517952642_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:52,887 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517952642_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:52,887 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517952642_0001_1_post
2016-05-17 20:45:52,887 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:52,887 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:52,887 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517952642_0001_00_000001, Exiting
2016-05-17 20:45:52,888 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517952642_0001_00_000001
2016-05-17 20:45:52,888 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517952642_0001_00_000001
2016-05-17 20:45:52,888 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517952642_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 20:45:52,890 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517952642_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:52,890 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517952642_0001_00_000001 completed successfully
2016-05-17 20:45:52,891 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517952642_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 20:45:52,895 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...m/testFilterAll/filterall, applicationId=application_1463517952642_0001
2016-05-17 20:45:52,898 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:52,899 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:52,899 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:52,899 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:52,899 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:52,899 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:52,900 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:52,900 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:52,900 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:52,901 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:52,901 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:52,901 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:52,902 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:52,905 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/698E15FA8FE04110888027F1EDE9B0F8/ADC63FB509664D6C9EDAB5343D3E587C/.tez/application_1463517952642_0001
2016-05-17 20:45:52,906 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall/_temporary
2016-05-17 20:45:52,906 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.285
2016-05-17 20:45:52,939 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testFilterAll/filterall
2016-05-17 20:45:52,963 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:52,963 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:52,986 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 20:45:52,986 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 20:45:53,002 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-05-17 20:45:53,018 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.030
2016-05-17 20:45:53,018 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:53,018 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:53,018 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:53,034 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt
2016-05-17 20:45:53,040 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown
2016-05-17 20:45:53,086 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 20:45:53,086 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 20:45:53,086 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown"]
2016-05-17 20:45:53,086 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 20:45:53,087 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 20:45:53,087 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 20:45:53,088 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...own/simplerelativeunknown
2016-05-17 20:45:53,088 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:53,099 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 20:45:53,104 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:53,104 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:53,104 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:53,113 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202/.tez/application_1463517953104_0001 doesn't exist and is created
2016-05-17 20:45:53,129 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:53,130 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:53,130 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202/.tez/application_1463517953104_0001
2016-05-17 20:45:53,133 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517953104_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:53,143 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:53,144 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:53,145 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:53,146 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:53,146 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202/.tez/application_1463517953104_0001/localmode-log-dir
2016-05-17 20:45:53,146 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202/.tez/application_1463517953104_0001/localmode-log-dir/history.txt.appattempt_1463517953104_0001_000000, maxErrors=10
2016-05-17 20:45:53,146 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:53,147 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517953104_0001_000000, appSubmitTime=1463517953133, launchTime=1463517953133
2016-05-17 20:45:53,155 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:53,156 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:53,157 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:53,157 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:53,163 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:49472
2016-05-17 20:45:53,165 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:53,166 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517953104_0001_000000, startTime=1463517953166
2016-05-17 20:45:53,166 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:53,166 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:53,230 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:53,230 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:53,230 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...own/simplerelativeunknown, applicationId=application_1463517953104_0001, dagName=[FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202] test/(1/1) ...own/simplerelativeunknown
2016-05-17 20:45:53,247 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202] test/(1/1) ...own/simplerelativeunknown
2016-05-17 20:45:53,248 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517953104_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202/.tez/application_1463517953104_0001/localmode-log-dir/dag_1463517953104_0001_1.dot
2016-05-17 20:45:53,249 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202/.tez/application_1463517953104_0001/localmode-log-dir/dag_1463517953104_0001_1-tez-dag.pb.txt
2016-05-17 20:45:53,264 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517953104_0001_1
2016-05-17 20:45:53,265 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:53,265 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202] test/(1/1) ...own/simplerelativeunknown
2016-05-17 20:45:53 Running Dag: dag_1463517953104_0001_1
2016-05-17 20:45:53,278 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:53,278 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517953104_0001_1, submitTime=1463517953247
2016-05-17 20:45:53,279 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F4FD1F2D4E2B402BA1A24AC7D23055F6
2016-05-17 20:45:53,279 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:53,279 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:53,280 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517953104_0001_1, initTime=1463517953279
2016-05-17 20:45:53,280 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517953104_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:53,280 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:DAG_STARTED]: dagID=dag_1463517953104_0001_1, startTime=1463517953280
2016-05-17 20:45:53,281 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:53,281 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517953104_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:53,281 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BEB67DFC5DAB4C0D930D58116C1C92B7 : {C9009DA06C044CBCABD8DC9B87438BBD={InputName=C9009DA06C044CBCABD8DC9B87438BBD}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:53,281 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: C9009DA06C044CBCABD8DC9B87438BBD, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:53,281 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7]
2016-05-17 20:45:53,281 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7]
2016-05-17 20:45:53,282 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7]
2016-05-17 20:45:53,282 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7]. Starting root input initializers: 1
2016-05-17 20:45:53,280 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:53,281 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...own/simplerelativeunknown, applicationId=application_1463517953104_0001, dagName=[FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202] test/(1/1) ...own/simplerelativeunknown
2016-05-17 20:45:53,282 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:53,282 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6]
2016-05-17 20:45:53,283 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: C9009DA06C044CBCABD8DC9B87438BBD on vertex vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7]
2016-05-17 20:45:53,284 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:53,285 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input C9009DA06C044CBCABD8DC9B87438BBD asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:53,285 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:53,285 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:53,285 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 20:45:53,286 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6]
2016-05-17 20:45:53,286 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6]
2016-05-17 20:45:53,286 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6]
2016-05-17 20:45:53,286 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=24E8A8ECAD434F3F811665337F82343F, vertexId=vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:53,286 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=24E8A8ECAD434F3F811665337F82343F, vertexId=vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6]
2016-05-17 20:45:53,287 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-05-17 20:45:53,288 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: C9009DA06C044CBCABD8DC9B87438BBD on vertex vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7]
2016-05-17 20:45:53,288 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=24E8A8ECAD434F3F811665337F82343F, vertexName=F4FD1F2D4E2B402BA1A24AC7D23055F6, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:53,288 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=24E8A8ECAD434F3F811665337F82343F, vertexId=vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6]
2016-05-17 20:45:53,289 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:53,289 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F4FD1F2D4E2B402BA1A24AC7D23055F6, vertexId=vertex_1463517953104_0001_1_01, initRequestedTime=1463517953282, initedTime=1463517953289, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 20:45:53,289 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6] transitioned from NEW to INITED due to event V_INIT
2016-05-17 20:45:53,290 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:53,290 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {C9009DA06C044CBCABD8DC9B87438BBD=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:53,290 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7] parallelism set to 1
2016-05-17 20:45:53,290 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:53,290 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517953104_0001_1_00, eventCount=1
2016-05-17 20:45:53,291 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BEB67DFC5DAB4C0D930D58116C1C92B7, vertexId=vertex_1463517953104_0001_1_00, initRequestedTime=1463517953281, initedTime=1463517953291, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:53,291 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7] with distanceFromRoot: 0
2016-05-17 20:45:53,291 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:53,291 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BEB67DFC5DAB4C0D930D58116C1C92B7
2016-05-17 20:45:53,291 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:53,291 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7]
2016-05-17 20:45:53,292 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:53,292 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517953104_0001_1_00, startRequestedTime=1463517953291, startedTime=1463517953291
2016-05-17 20:45:53,292 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:53,293 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:53,293 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:TASK_STARTED]: vertexName=BEB67DFC5DAB4C0D930D58116C1C92B7, taskId=task_1463517953104_0001_1_00_000000, scheduledTime=1463517953292, launchTime=1463517953292
2016-05-17 20:45:53,293 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953104_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:53,293 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463517953104_0001_1_00 for vertex: vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6] numStartedSources: 1 numSources: 1
2016-05-17 20:45:53,293 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6] with distanceFromRoot: 1
2016-05-17 20:45:53,293 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7]
2016-05-17 20:45:53,293 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: BEB67DFC5DAB4C0D930D58116C1C92B7 in vertex: F4FD1F2D4E2B402BA1A24AC7D23055F6
2016-05-17 20:45:53,294 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F4FD1F2D4E2B402BA1A24AC7D23055F6 with 1 source tasks and 1 pending tasks
2016-05-17 20:45:53,294 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:53,294 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517953104_0001_1_01, startRequestedTime=1463517953293, startedTime=1463517953293
2016-05-17 20:45:53,294 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:53,294 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202] test/(1/1) ...own/simplerelativeunknown, VertexName: BEB67DFC5DAB4C0D930D58116C1C92B7, VertexParallelism: 1, TaskAttemptID:attempt_1463517953104_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C9009DA06C044CBCABD8DC9B87438BBD, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F4FD1F2D4E2B402BA1A24AC7D23055F6, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:53,294 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953104_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:53,295 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:53,296 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:53,296 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:53,296 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:53,297 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517953104_0001_00_000001
2016-05-17 20:45:53,297 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953104_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:53,298 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517953104_0001_00_000001
2016-05-17 20:45:53,299 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:53,299 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463517953104_0001, with dag id: dag_1463517953104_0001_1
2016-05-17 20:45:53,300 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953104_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:53,300 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:53,300 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517953104_0001_00_000001, launchTime=1463517953300
2016-05-17 20:45:53,301 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517953104_0001_00_000001
2016-05-17 20:45:53,302 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517953104_0001_1_00_000000_0] to container: [container_1463517953104_0001_00_000001]
2016-05-17 20:45:53,302 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953104_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:53,302 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517953104_0001_00_000001 given task: attempt_1463517953104_0001_1_00_000000_0
2016-05-17 20:45:53,302 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517953104_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517953104_0001_1_00_000000_0
2016-05-17 20:45:53,303 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517953104_0001_1_00_000000_0] started. Is using containerId: [container_1463517953104_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:53,303 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BEB67DFC5DAB4C0D930D58116C1C92B7, taskAttemptId=attempt_1463517953104_0001_1_00_000000_0, startTime=1463517953302, containerId=container_1463517953104_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517953104_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:53,303 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:53,303 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953104_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:53,303 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953104_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:53 Starting to run new task attempt: attempt_1463517953104_0001_1_00_000000_0
2016-05-17 20:45:53,304 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517953104_0001_1_00_000000_0
2016-05-17 20:45:53,304 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:53,304 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:53,304 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:53,304 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:53,304 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:53,305 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:53,305 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202] test/(1/1) ...own/simplerelativeunknown, VertexName: BEB67DFC5DAB4C0D930D58116C1C92B7, VertexParallelism: 1, TaskAttemptID:attempt_1463517953104_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C9009DA06C044CBCABD8DC9B87438BBD, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F4FD1F2D4E2B402BA1A24AC7D23055F6, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:53,305 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:53,310 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517953104_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:53,312 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517953104_0001_1_00_000000_0
2016-05-17 20:45:53,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:53,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:53,313 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C9009DA06C044CBCABD8DC9B87438BBD, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:53,313 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:53,313 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C9009DA06C044CBCABD8DC9B87438BBD
2016-05-17 20:45:53,314 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: C9009DA06C044CBCABD8DC9B87438BBD
2016-05-17 20:45:53,314 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:53,316 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:53,316 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: C9009DA06C044CBCABD8DC9B87438BBD
2016-05-17 20:45:53,316 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C9009DA06C044CBCABD8DC9B87438BBD
2016-05-17 20:45:53,316 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F4FD1F2D4E2B402BA1A24AC7D23055F6, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 20:45:53,316 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:53,317 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F4FD1F2D4E2B402BA1A24AC7D23055F6
2016-05-17 20:45:53,317 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:53,318 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 20:45:53,318 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F4FD1F2D4E2B402BA1A24AC7D23055F6
2016-05-17 20:45:53,319 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F4FD1F2D4E2B402BA1A24AC7D23055F6
2016-05-17 20:45:53,327 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BEB67DFC5DAB4C0D930D58116C1C92B7, ordinal: 0
2016-05-17 20:45:53,327 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BEB67DFC5DAB4C0D930D58116C1C92B7, mem on start (mb), free: 258, total: 541, max: 672
2016-05-17 20:45:53,327 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:53,327 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:53,327 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:53,327 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:53,327 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:53,328 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:53,328 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:53,328 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 20:45:53,328 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:53,328 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 20:45:53,328 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C9009DA06C044CBCABD8DC9B87438BBD, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:53,328 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F4FD1F2D4E2B402BA1A24AC7D23055F6, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 20:45:53,329 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:53,329 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C9009DA06C044CBCABD8DC9B87438BBD being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:53,329 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:53,329 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:53,329 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C9009DA06C044CBCABD8DC9B87438BBD
2016-05-17 20:45:53,329 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:53,329 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C9009DA06C044CBCABD8DC9B87438BBD
2016-05-17 20:45:53,330 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:53,330 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517953104_0001_1_00_000000_0
2016-05-17 20:45:53,330 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:53,330 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1015fc27
2016-05-17 20:45:53,331 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1015fc27
2016-05-17 20:45:53,332 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:53,332 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:53,333 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 20:45:53,333 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"] streamed: true, id: C9009DA06C044CBCABD8DC9B87438BBD
2016-05-17 20:45:53,333 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:-1]], id: 33A04523F6CE49278B94AF86B1734C29
2016-05-17 20:45:53,333 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:-1]] 33A04523F6CE49278B94AF86B1734C29
2016-05-17 20:45:53,335 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@3cfe7c4c; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@679280dc; comparator=cascading.tuple.hadoop.util.TupleComparator@434a20a3; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 20:45:53,335 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 20:45:53,335 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 20:45:53,335 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:53,359 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 20:45:53,359 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 20:45:53,359 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 20:45:53,359 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 20:45:53,359 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 20:45:53,360 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 20:45:53,360 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:53,360 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BEB67DFC5DAB4C0D930D58116C1C92B7, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:53,362 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517953104_0001_1_00_000000_0
2016-05-17 20:45:53,362 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BEB67DFC5DAB4C0D930D58116C1C92B7, mem on close (mb), free: 322, total: 554, max: 672
2016-05-17 20:45:53,363 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 20:45:53,363 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 20:45:53,363 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-05-17 20:45:53,363 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-05-17 20:45:53,365 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 20:45:53 Completed running task attempt: attempt_1463517953104_0001_1_00_000000_0
2016-05-17 20:45:53,368 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517953104_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:53,369 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517953104_0001_1_00, eventCount=1
2016-05-17 20:45:53,370 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BEB67DFC5DAB4C0D930D58116C1C92B7, taskAttemptId=attempt_1463517953104_0001_1_00_000000_0, startTime=1463517953302, finishTime=1463517953370, timeTaken=68, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517953360, Process_Duration=2, Process_End_Time=1463517953362, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 20:45:53,370 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953104_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:53,370 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:53,370 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517953104_0001_1_00_000000_0
2016-05-17 20:45:53,371 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:53,371 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:TASK_FINISHED]: vertexName=BEB67DFC5DAB4C0D930D58116C1C92B7, taskId=task_1463517953104_0001_1_00_000000, startTime=1463517953302, finishTime=1463517953371, timeTaken=69, status=SUCCEEDED, successfulAttemptID=attempt_1463517953104_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517953360, Process_Duration=2, Process_End_Time=1463517953362, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 20:45:53,371 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953104_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:53,372 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7] : 1
2016-05-17 20:45:53,372 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7]
2016-05-17 20:45:53,373 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:53,373 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:VERTEX_FINISHED]: vertexName=BEB67DFC5DAB4C0D930D58116C1C92B7, vertexId=vertex_1463517953104_0001_1_00, initRequestedTime=1463517953281, initedTime=1463517953291, startRequestedTime=1463517953291, startedTime=1463517953291, finishTime=1463517953372, timeTaken=81, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517953360, Process_Duration=2, Process_End_Time=1463517953362, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463517953302, firstTasksToStart=[ task_1463517953104_0001_1_00_000000 ], lastTaskFinishTime=1463517953370, lastTasksToFinish=[ task_1463517953104_0001_1_00_000000 ], minTaskDuration=68, maxTaskDuration=68, avgTaskDuration=68.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517953104_0001_1_00_000000 ], longestDurationTasks=[ task_1463517953104_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:53,373 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:53,374 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6] attempt: attempt_1463517953104_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 20:45:53,374 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F4FD1F2D4E2B402BA1A24AC7D23055F6
2016-05-17 20:45:53,374 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6]
2016-05-17 20:45:53,374 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:53,374 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:TASK_STARTED]: vertexName=F4FD1F2D4E2B402BA1A24AC7D23055F6, taskId=task_1463517953104_0001_1_01_000000, scheduledTime=1463517953374, launchTime=1463517953374
2016-05-17 20:45:53,375 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953104_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:53,375 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6]
2016-05-17 20:45:53,375 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202] test/(1/1) ...own/simplerelativeunknown, VertexName: F4FD1F2D4E2B402BA1A24AC7D23055F6, VertexParallelism: 1, TaskAttemptID:attempt_1463517953104_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BEB67DFC5DAB4C0D930D58116C1C92B7, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=24E8A8ECAD434F3F811665337F82343F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:53,373 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:53,375 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:53,376 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953104_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:53,378 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953104_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:53,378 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463517953360, Process_Duration=2, Process_End_Time=1463517953362, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-17 20:45:53,378 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:53,378 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:53,379 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:53,379 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517953104_0001_00_000001
2016-05-17 20:45:53,379 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:53,379 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517953104_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:53,379 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517953104_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:53,379 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517953104_0001_00_000001, Exiting
2016-05-17 20:45:53,380 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517953104_0001_00_000001
2016-05-17 20:45:53,380 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517953104_0001_00_000001
2016-05-17 20:45:53,379 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517953104_0001_00_000001, stoppedTime=1463517953379, exitStatus=0
2016-05-17 20:45:53,381 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517953104_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:53,381 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953104_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:53,382 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517953104_0001_00_000001 completed successfully
2016-05-17 20:45:53,382 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517953104_0001_00_000002
2016-05-17 20:45:53,382 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953104_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:53,382 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953104_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:53,382 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517953104_0001_00_000001
2016-05-17 20:45:53,383 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517953104_0001_00_000002
2016-05-17 20:45:53,384 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 20:45:53,385 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953104_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:53,386 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:53,386 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517953104_0001_00_000002, launchTime=1463517953386
2016-05-17 20:45:53,386 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517953104_0001_00_000002
2016-05-17 20:45:53,387 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517953104_0001_1_01_000000_0] to container: [container_1463517953104_0001_00_000002]
2016-05-17 20:45:53,387 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953104_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:53,387 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517953104_0001_00_000002 given task: attempt_1463517953104_0001_1_01_000000_0
2016-05-17 20:45:53,387 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517953104_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517953104_0001_1_01_000000_0
2016-05-17 20:45:53,388 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517953104_0001_1_01_000000_0] started. Is using containerId: [container_1463517953104_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 20:45:53 Starting to run new task attempt: attempt_1463517953104_0001_1_01_000000_0
2016-05-17 20:45:53,388 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F4FD1F2D4E2B402BA1A24AC7D23055F6, taskAttemptId=attempt_1463517953104_0001_1_01_000000_0, startTime=1463517953387, containerId=container_1463517953104_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517953104_0001_00_000002/teamcity, completedLogs=
2016-05-17 20:45:53,388 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517953104_0001_1_01_000000_0
2016-05-17 20:45:53,388 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953104_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:53,388 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:53,388 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:53,388 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:53,388 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:53,389 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:53,389 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:53,389 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:53,389 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202] test/(1/1) ...own/simplerelativeunknown, VertexName: F4FD1F2D4E2B402BA1A24AC7D23055F6, VertexParallelism: 1, TaskAttemptID:attempt_1463517953104_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BEB67DFC5DAB4C0D930D58116C1C92B7, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=24E8A8ECAD434F3F811665337F82343F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:53,388 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953104_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:53,389 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:53,389 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517953104_0001_1_01_000000_0
2016-05-17 20:45:53,389 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:53,390 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:53,395 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517953104_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 20:45:53,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:53,396 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=24E8A8ECAD434F3F811665337F82343F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:53,397 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:53,397 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 24E8A8ECAD434F3F811665337F82343F
2016-05-17 20:45:53,397 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:53,397 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 24E8A8ECAD434F3F811665337F82343F
2016-05-17 20:45:53,397 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BEB67DFC5DAB4C0D930D58116C1C92B7, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 20:45:53,398 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:53,398 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BEB67DFC5DAB4C0D930D58116C1C92B7
2016-05-17 20:45:53,398 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:53,399 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 20:45:53,399 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 20:45:53,400 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: BEB67DFC5DAB4C0D930D58116C1C92B7
2016-05-17 20:45:53,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BEB67DFC5DAB4C0D930D58116C1C92B7
2016-05-17 20:45:53,406 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F4FD1F2D4E2B402BA1A24AC7D23055F6, ordinal: 1
2016-05-17 20:45:53,406 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F4FD1F2D4E2B402BA1A24AC7D23055F6, mem on start (mb), free: 315, total: 554, max: 672
2016-05-17 20:45:53,407 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:53,407 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:53,407 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:53,408 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:53,409 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 24E8A8ECAD434F3F811665337F82343F
2016-05-17 20:45:53,409 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:53,409 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:53,409 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:53,409 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:53,409 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 20:45:53,409 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:53,409 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 20:45:53,410 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 24E8A8ECAD434F3F811665337F82343F, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:53,410 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BEB67DFC5DAB4C0D930D58116C1C92B7, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 20:45:53,410 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:53,410 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BEB67DFC5DAB4C0D930D58116C1C92B7 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:53,410 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:53,410 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BEB67DFC5DAB4C0D930D58116C1C92B7
2016-05-17 20:45:53,410 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:53,411 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:53,411 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 20:45:53,411 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: BEB67DFC5DAB4C0D930D58116C1C92B7 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 20:45:53,411 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 20:45:53,412 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 20:45:53,412 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 20:45:53,413 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BEB67DFC5DAB4C0D930D58116C1C92B7
2016-05-17 20:45:53,413 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:53,413 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517953104_0001_1_01_000000_0
2016-05-17 20:45:53,413 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463517953104_0001_1_00_000000_0_10049, runDuration: 50365, ]
2016-05-17 20:45:53,415 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463517953104_0001_1_00_000000_0_10049] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1463517953415, TimeTaken=1, Rate=0.09 MB/s
2016-05-17 20:45:53,415 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : BEB67DFC5DAB4C0D930D58116C1C92B7
2016-05-17 20:45:53,415 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 20:45:53,415 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [BEB67DFC5DAB4C0D930D58116C1C92B7] #8 in 1ms
2016-05-17 20:45:53,416 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [BEB67DFC5DAB4C0D930D58116C1C92B7] #8, status:false, isInterrupted:false
2016-05-17 20:45:53,416 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:-1]]
2016-05-17 20:45:53,416 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 20:45:53,417 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:-1]] streamed: true, id: 33A04523F6CE49278B94AF86B1734C29
2016-05-17 20:45:53,417 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-05-17 20:45:53,417 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown"], id: 24E8A8ECAD434F3F811665337F82343F
2016-05-17 20:45:53,417 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:53,417 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 20:45:53,417 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown"]
2016-05-17 20:45:53,418 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-17 20:45:53,418 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : BEB67DFC5DAB4C0D930D58116C1C92B7
2016-05-17 20:45:53,418 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 20:45:53,418 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:-1]] 33A04523F6CE49278B94AF86B1734C29, for 1 inputs
2016-05-17 20:45:53,418 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F4FD1F2D4E2B402BA1A24AC7D23055F6, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:53,420 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@53360449; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@7f537f4c
2016-05-17 20:45:53,422 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:53,422 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:53,422 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517953104_0001_1_01_000000_0
2016-05-17 20:45:53,423 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517953104_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 20:45:53,423 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517953104_0001_1_01_000000_0
2016-05-17 20:45:53,424 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F4FD1F2D4E2B402BA1A24AC7D23055F6, mem on close (mb), free: 313, total: 554, max: 672
2016-05-17 20:45:53,424 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: BEB67DFC5DAB4C0D930D58116C1C92B7
2016-05-17 20:45:53,424 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 20:45:53,425 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 20:45:53,425 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:53,425 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 20:45:53,425 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 20:45:53 Completed running task attempt: attempt_1463517953104_0001_1_01_000000_0
2016-05-17 20:45:53,426 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517953104_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:53,427 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=127, FILE_BYTES_WRITTEN=32, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463517953419, Process_Duration=4, Process_End_Time=1463517953423, Read_Duration=1, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-05-17 20:45:53,427 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:53,428 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:53,428 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:53,428 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517953104_0001_00_000002
2016-05-17 20:45:53,427 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F4FD1F2D4E2B402BA1A24AC7D23055F6, taskAttemptId=attempt_1463517953104_0001_1_01_000000_0, startTime=1463517953387, finishTime=1463517953427, timeTaken=40, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=127, FILE_BYTES_WRITTEN=32, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517953419, Process_Duration=4, Process_End_Time=1463517953423, Read_Duration=1, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-17 20:45:53,429 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953104_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:53,429 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:53,429 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517953104_0001_1_01_000000_0
2016-05-17 20:45:53,429 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:53,430 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:TASK_FINISHED]: vertexName=F4FD1F2D4E2B402BA1A24AC7D23055F6, taskId=task_1463517953104_0001_1_01_000000, startTime=1463517953387, finishTime=1463517953429, timeTaken=42, status=SUCCEEDED, successfulAttemptID=attempt_1463517953104_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=127, FILE_BYTES_WRITTEN=32, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517953419, Process_Duration=4, Process_End_Time=1463517953423, Read_Duration=1, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-17 20:45:53,430 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953104_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:53,430 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6] : 1
2016-05-17 20:45:53,430 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6]
2016-05-17 20:45:53,431 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:53,431 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:VERTEX_FINISHED]: vertexName=F4FD1F2D4E2B402BA1A24AC7D23055F6, vertexId=vertex_1463517953104_0001_1_01, initRequestedTime=1463517953282, initedTime=1463517953289, startRequestedTime=1463517953293, startedTime=1463517953293, finishTime=1463517953430, timeTaken=137, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=127, FILE_BYTES_WRITTEN=32, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=580911104, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517953419, Process_Duration=4, Process_End_Time=1463517953423, Read_Duration=1, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1463517953387, firstTasksToStart=[ task_1463517953104_0001_1_01_000000 ], lastTaskFinishTime=1463517953427, lastTasksToFinish=[ task_1463517953104_0001_1_01_000000 ], minTaskDuration=40, maxTaskDuration=40, avgTaskDuration=40.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517953104_0001_1_01_000000 ], longestDurationTasks=[ task_1463517953104_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:53,431 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:53,431 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953104_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:53,432 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517953104_0001_00_000002, stoppedTime=1463517953432, exitStatus=0
2016-05-17 20:45:53,432 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953104_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:53,432 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:53,432 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517953104_0001_1
2016-05-17 20:45:53,433 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517953104_0001_1
2016-05-17 20:45:53,433 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463517953104_0001_1_00 [BEB67DFC5DAB4C0D930D58116C1C92B7]
2016-05-17 20:45:53,433 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 24E8A8ECAD434F3F811665337F82343F for vertex: vertex_1463517953104_0001_1_01 [F4FD1F2D4E2B402BA1A24AC7D23055F6]
2016-05-17 20:45:53,434 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:53,439 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517953104_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 20:45:53,439 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517953104_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:53,439 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517953104_0001_00_000002, Exiting
2016-05-17 20:45:53,440 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517953104_0001_00_000002
2016-05-17 20:45:53,440 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517953104_0001_00_000002
2016-05-17 20:45:53,440 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517953104_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:53,440 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517953104_0001_00_000002 completed successfully
2016-05-17 20:45:53,442 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517953104_0001_1, queueSize=0
2016-05-17 20:45:53,442 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:53,443 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953104_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517953104_0001_1, startTime=1463517953280, finishTime=1463517953440, timeTaken=160, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=189, FILE_BYTES_WRITTEN=159, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1161822208, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927035906779, Process_Duration=6, Process_End_Time=2927035906785, Read_Duration=3, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-05-17 20:45:53,443 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517953104_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:53,444 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517953104_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:53,444 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953104_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:53,444 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517953104_0001_00_000002
2016-05-17 20:45:53 Completed Dag: dag_1463517953104_0001_1
2016-05-17 20:45:53,446 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517953104_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:53,447 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517953104_0001_1_post
2016-05-17 20:45:53,447 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:53,447 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:53,451 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...own/simplerelativeunknown, applicationId=application_1463517953104_0001
2016-05-17 20:45:53,453 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:53,453 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:53,453 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:53,453 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:53,453 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:53,453 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:53,454 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:53,454 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:53,454 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:53,454 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:53,454 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:53,455 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:53,455 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:53,459 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FA08973B86FF46D2B175C362C76F558B/E13F370E38BC46E8BEA1B79BAF111202/.tez/application_1463517953104_0001
2016-05-17 20:45:53,460 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown/_temporary
2016-05-17 20:45:53,460 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.373
2016-05-17 20:45:53,515 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown
2016-05-17 20:45:53,543 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:53,543 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:53,588 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.022
2016-05-17 20:45:53,606 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 20:45:53,606 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 20:45:53,615 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.049
2016-05-17 20:45:53,615 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-17 20:45:53,616 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-17 20:45:53,616 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:53,639 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt
2016-05-17 20:45:53,646 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt
2016-05-17 20:45:53,652 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup
2016-05-17 20:45:53,753 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] starting
2016-05-17 20:45:53,754 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-05-17 20:45:53,754 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 20:45:53,754 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup"]
2016-05-17 20:45:53,755 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] parallel execution of steps is enabled: false
2016-05-17 20:45:53,755 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executing total steps: 1
2016-05-17 20:45:53,755 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] allocating management threads: 1
2016-05-17 20:45:53,756 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] starting step: (1/1) ...estCoGroup/complexcogroup
2016-05-17 20:45:53,756 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [lower*upper] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:53,768 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tez session mode enabled: true
2016-05-17 20:45:53,772 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:53,773 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:53,773 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:53,783 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24/.tez/application_1463517953773_0001 doesn't exist and is created
2016-05-17 20:45:53,799 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:53,799 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:53,800 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24/.tez/application_1463517953773_0001
2016-05-17 20:45:53,804 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517953773_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:53,819 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:53,820 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:53,823 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:53,823 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:53,823 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24/.tez/application_1463517953773_0001/localmode-log-dir
2016-05-17 20:45:53,824 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24/.tez/application_1463517953773_0001/localmode-log-dir/history.txt.appattempt_1463517953773_0001_000000, maxErrors=10
2016-05-17 20:45:53,824 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:53,825 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517953773_0001_000000, appSubmitTime=1463517953804, launchTime=1463517953804
2016-05-17 20:45:53,830 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:53,833 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:53,833 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:53,834 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:53,838 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:56526
2016-05-17 20:45:53,840 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:53,841 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517953773_0001_000000, startTime=1463517953841
2016-05-17 20:45:53,842 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:53,842 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:53,899 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:53,900 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:53,900 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...estCoGroup/complexcogroup, applicationId=application_1463517953773_0001, dagName=[B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-05-17 20:45:53,919 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-05-17 20:45:53,920 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517953773_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24/.tez/application_1463517953773_0001/localmode-log-dir/dag_1463517953773_0001_1.dot
2016-05-17 20:45:53,921 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24/.tez/application_1463517953773_0001/localmode-log-dir/dag_1463517953773_0001_1-tez-dag.pb.txt
2016-05-17 20:45:53,931 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517953773_0001_1
2016-05-17 20:45:53,932 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:53,932 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-05-17 20:45:53 Running Dag: dag_1463517953773_0001_1
2016-05-17 20:45:53,946 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517953773_0001_1, submitTime=1463517953919
2016-05-17 20:45:53,946 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:53,947 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FEBA4EBF8BDB40D78748DD504B2E84DE
2016-05-17 20:45:53,947 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:53,948 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517953773_0001_1, initTime=1463517953946
2016-05-17 20:45:53,948 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:53,948 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517953773_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:53,949 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:DAG_STARTED]: dagID=dag_1463517953773_0001_1, startTime=1463517953949
2016-05-17 20:45:53,949 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:53,949 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...estCoGroup/complexcogroup, applicationId=application_1463517953773_0001, dagName=[B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-05-17 20:45:53,950 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517953773_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:53,950 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F7C9F8290A98443ABC86347D3CDCB5E3 : {6837B65D2EA7478B8D296BC295D4907B={InputName=6837B65D2EA7478B8D296BC295D4907B}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:53,950 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 6837B65D2EA7478B8D296BC295D4907B, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:53,950 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3]
2016-05-17 20:45:53,950 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3]
2016-05-17 20:45:53,950 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3]
2016-05-17 20:45:53,951 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3]. Starting root input initializers: 1
2016-05-17 20:45:53,951 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:53,951 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B37A4A0444944283B5C65E2C19FC35BA : {EA413A1D8AA244E1B9EF6B102662BC59={InputName=EA413A1D8AA244E1B9EF6B102662BC59}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:53,951 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: EA413A1D8AA244E1B9EF6B102662BC59, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:53,951 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA]
2016-05-17 20:45:53,952 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA]
2016-05-17 20:45:53,952 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA]
2016-05-17 20:45:53,952 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA]. Starting root input initializers: 1
2016-05-17 20:45:53,951 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:53,953 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 6837B65D2EA7478B8D296BC295D4907B on vertex vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3]
2016-05-17 20:45:53,954 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:53,955 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: EA413A1D8AA244E1B9EF6B102662BC59 on vertex vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA]
2016-05-17 20:45:53,955 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE]
2016-05-17 20:45:53,956 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:53,957 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 6837B65D2EA7478B8D296BC295D4907B asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:53,957 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:53,958 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:53,958 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 20:45:53,958 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE]
2016-05-17 20:45:53,959 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE]
2016-05-17 20:45:53,959 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE]
2016-05-17 20:45:53,960 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=763782EE94B642ECB31B619719C37B70, vertexId=vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:53,960 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=763782EE94B642ECB31B619719C37B70, vertexId=vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE]
2016-05-17 20:45:53,962 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:53,962 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=763782EE94B642ECB31B619719C37B70, vertexName=FEBA4EBF8BDB40D78748DD504B2E84DE, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:53,962 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input EA413A1D8AA244E1B9EF6B102662BC59 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:53,963 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=763782EE94B642ECB31B619719C37B70, vertexId=vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE]
2016-05-17 20:45:53,965 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-05-17 20:45:53,965 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 6837B65D2EA7478B8D296BC295D4907B on vertex vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3]
2016-05-17 20:45:53,965 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:53,966 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:53,966 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FEBA4EBF8BDB40D78748DD504B2E84DE, vertexId=vertex_1463517953773_0001_1_02, initRequestedTime=1463517953955, initedTime=1463517953965, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 20:45:53,966 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE] transitioned from NEW to INITED due to event V_INIT
2016-05-17 20:45:53,967 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:53,967 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:53,969 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {6837B65D2EA7478B8D296BC295D4907B=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:53,969 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3] parallelism set to 1
2016-05-17 20:45:53,970 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:53,970 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517953773_0001_1_00, eventCount=1
2016-05-17 20:45:53,967 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] submitted tez dag to app master: application_1463517953773_0001, with dag id: dag_1463517953773_0001_1
2016-05-17 20:45:53,970 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-05-17 20:45:53,970 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F7C9F8290A98443ABC86347D3CDCB5E3, vertexId=vertex_1463517953773_0001_1_00, initRequestedTime=1463517953950, initedTime=1463517953970, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:53,972 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3] with distanceFromRoot: 0
2016-05-17 20:45:53,972 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:53,970 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:53,972 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F7C9F8290A98443ABC86347D3CDCB5E3
2016-05-17 20:45:53,972 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3]
2016-05-17 20:45:53,972 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: EA413A1D8AA244E1B9EF6B102662BC59 on vertex vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA]
2016-05-17 20:45:53,973 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:53,973 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517953773_0001_1_00, startRequestedTime=1463517953972, startedTime=1463517953972
2016-05-17 20:45:53,975 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:53,975 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:53,975 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:TASK_STARTED]: vertexName=F7C9F8290A98443ABC86347D3CDCB5E3, taskId=task_1463517953773_0001_1_00_000000, scheduledTime=1463517953975, launchTime=1463517953975
2016-05-17 20:45:53,976 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953773_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:53,976 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:53,976 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {EA413A1D8AA244E1B9EF6B102662BC59=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:53,977 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA] parallelism set to 1
2016-05-17 20:45:53,977 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:53,977 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517953773_0001_1_01, eventCount=1
2016-05-17 20:45:53,978 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B37A4A0444944283B5C65E2C19FC35BA, vertexId=vertex_1463517953773_0001_1_01, initRequestedTime=1463517953951, initedTime=1463517953978, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:53,978 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA] with distanceFromRoot: 0
2016-05-17 20:45:53,978 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:53,980 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:53,981 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463517953773_0001_1_00 for vertex: vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE] numStartedSources: 1 numSources: 2
2016-05-17 20:45:53,981 INFO impl.VertexImpl (VertexImpl.java:transition(3160)) - Cannot start vertex: vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE] numStartedSources: 1 numSources: 2
2016-05-17 20:45:53,981 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3]
2016-05-17 20:45:53,982 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B37A4A0444944283B5C65E2C19FC35BA
2016-05-17 20:45:53,982 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA]
2016-05-17 20:45:53,982 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517953773_0001_1_01, startRequestedTime=1463517953982, startedTime=1463517953982
2016-05-17 20:45:53,982 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:53,985 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:53,986 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: F7C9F8290A98443ABC86347D3CDCB5E3, VertexParallelism: 1, TaskAttemptID:attempt_1463517953773_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=6837B65D2EA7478B8D296BC295D4907B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FEBA4EBF8BDB40D78748DD504B2E84DE, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:53,986 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953773_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:53,986 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:53,987 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:TASK_STARTED]: vertexName=B37A4A0444944283B5C65E2C19FC35BA, taskId=task_1463517953773_0001_1_01_000000, scheduledTime=1463517953986, launchTime=1463517953986
2016-05-17 20:45:53,987 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953773_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:53,987 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463517953773_0001_1_01 for vertex: vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE] numStartedSources: 2 numSources: 2
2016-05-17 20:45:53,987 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE] with distanceFromRoot: 1
2016-05-17 20:45:53,987 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:53,988 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:53,987 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA]
2016-05-17 20:45:53,989 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B37A4A0444944283B5C65E2C19FC35BA in vertex: FEBA4EBF8BDB40D78748DD504B2E84DE
2016-05-17 20:45:53,990 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F7C9F8290A98443ABC86347D3CDCB5E3 in vertex: FEBA4EBF8BDB40D78748DD504B2E84DE
2016-05-17 20:45:53,990 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: FEBA4EBF8BDB40D78748DD504B2E84DE with 2 source tasks and 1 pending tasks
2016-05-17 20:45:53,990 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:53,990 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517953773_0001_1_02, startRequestedTime=1463517953989, startedTime=1463517953989
2016-05-17 20:45:53,991 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:53,991 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:53,992 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:53,993 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517953773_0001_00_000001
2016-05-17 20:45:53,993 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:53,993 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: B37A4A0444944283B5C65E2C19FC35BA, VertexParallelism: 1, TaskAttemptID:attempt_1463517953773_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EA413A1D8AA244E1B9EF6B102662BC59, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FEBA4EBF8BDB40D78748DD504B2E84DE, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:53,993 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953773_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:53,994 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:53,995 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517953773_0001_00_000001
2016-05-17 20:45:53,995 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:53,996 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:53,997 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517953773_0001_00_000001, launchTime=1463517953996
2016-05-17 20:45:53,997 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517953773_0001_00_000001
2016-05-17 20:45:53,998 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517953773_0001_00_000002
2016-05-17 20:45:53,998 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:53,999 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:54,000 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517953773_0001_00_000002
2016-05-17 20:45:54,000 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:54,003 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:54,004 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:54,024 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517953773_0001_00_000002, launchTime=1463517954002
2016-05-17 20:45:54,024 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517953773_0001_1_00_000000_0] to container: [container_1463517953773_0001_00_000001]
2016-05-17 20:45:54,024 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:54,024 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517953773_0001_00_000001 given task: attempt_1463517953773_0001_1_00_000000_0
2016-05-17 20:45:54,025 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517953773_0001_00_000001: 27 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517953773_0001_1_00_000000_0
2016-05-17 20:45:54,025 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517953773_0001_1_00_000000_0] started. Is using containerId: [container_1463517953773_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:54,026 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F7C9F8290A98443ABC86347D3CDCB5E3, taskAttemptId=attempt_1463517953773_0001_1_00_000000_0, startTime=1463517954025, containerId=container_1463517953773_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517953773_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:54,026 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953773_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:54,026 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953773_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:54 Starting to run new task attempt: attempt_1463517953773_0001_1_00_000000_0
2016-05-17 20:45:54,027 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517953773_0001_1_00_000000_0
2016-05-17 20:45:54,027 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:54,027 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:54,027 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:54,027 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:54,027 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:54,027 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:54,027 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:54,027 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: F7C9F8290A98443ABC86347D3CDCB5E3, VertexParallelism: 1, TaskAttemptID:attempt_1463517953773_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=6837B65D2EA7478B8D296BC295D4907B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FEBA4EBF8BDB40D78748DD504B2E84DE, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:54,028 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:54,032 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517953773_0001_00_000002
2016-05-17 20:45:54,038 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517953773_0001_1_00_000000_0
2016-05-17 20:45:54,039 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:54,039 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:54,040 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=6837B65D2EA7478B8D296BC295D4907B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:54,040 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:54,040 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 6837B65D2EA7478B8D296BC295D4907B
2016-05-17 20:45:54,040 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 6837B65D2EA7478B8D296BC295D4907B
2016-05-17 20:45:54,041 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:54,060 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517953773_0001_1_01_000000_0] to container: [container_1463517953773_0001_00_000002]
2016-05-17 20:45:54,060 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:54,060 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517953773_0001_1_01_000000_0] started. Is using containerId: [container_1463517953773_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 20:45:54,061 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B37A4A0444944283B5C65E2C19FC35BA, taskAttemptId=attempt_1463517953773_0001_1_01_000000_0, startTime=1463517954060, containerId=container_1463517953773_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517953773_0001_00_000002/teamcity, completedLogs=
2016-05-17 20:45:54,061 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953773_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:54,061 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953773_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:54,061 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517953773_0001_00_000002 given task: attempt_1463517953773_0001_1_01_000000_0
2016-05-17 20:45:54,061 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517953773_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517953773_0001_1_01_000000_0
2016-05-17 20:45:54,061 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:54 Starting to run new task attempt: attempt_1463517953773_0001_1_01_000000_0
2016-05-17 20:45:54,062 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517953773_0001_1_01_000000_0
2016-05-17 20:45:54,062 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:54,062 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:54,062 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:54,062 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:54,062 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:54,063 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:54,063 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: B37A4A0444944283B5C65E2C19FC35BA, VertexParallelism: 1, TaskAttemptID:attempt_1463517953773_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EA413A1D8AA244E1B9EF6B102662BC59, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FEBA4EBF8BDB40D78748DD504B2E84DE, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:54,063 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:54,063 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FEBA4EBF8BDB40D78748DD504B2E84DE, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 20:45:54,064 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:54,064 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:54,064 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FEBA4EBF8BDB40D78748DD504B2E84DE
2016-05-17 20:45:54,066 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 20:45:54,066 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: FEBA4EBF8BDB40D78748DD504B2E84DE
2016-05-17 20:45:54,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FEBA4EBF8BDB40D78748DD504B2E84DE
2016-05-17 20:45:54,069 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:54,068 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517953773_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:54,068 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517953773_0001_1_01_000000_0
2016-05-17 20:45:54,072 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:54,072 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:54,072 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 6837B65D2EA7478B8D296BC295D4907B
2016-05-17 20:45:54,072 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 6837B65D2EA7478B8D296BC295D4907B
2016-05-17 20:45:54,073 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=EA413A1D8AA244E1B9EF6B102662BC59, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:54,073 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517953773_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 20:45:54,074 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:54,074 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: EA413A1D8AA244E1B9EF6B102662BC59
2016-05-17 20:45:54,074 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: EA413A1D8AA244E1B9EF6B102662BC59
2016-05-17 20:45:54,075 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:54,076 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FEBA4EBF8BDB40D78748DD504B2E84DE, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 20:45:54,076 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:54,076 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FEBA4EBF8BDB40D78748DD504B2E84DE
2016-05-17 20:45:54,078 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 20:45:54,078 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: FEBA4EBF8BDB40D78748DD504B2E84DE
2016-05-17 20:45:54,078 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FEBA4EBF8BDB40D78748DD504B2E84DE
2016-05-17 20:45:54,080 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:54,082 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:54,082 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: EA413A1D8AA244E1B9EF6B102662BC59
2016-05-17 20:45:54,083 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: EA413A1D8AA244E1B9EF6B102662BC59
2016-05-17 20:45:54,087 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F7C9F8290A98443ABC86347D3CDCB5E3, ordinal: 1
2016-05-17 20:45:54,088 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F7C9F8290A98443ABC86347D3CDCB5E3, mem on start (mb), free: 416, total: 553, max: 672
2016-05-17 20:45:54,088 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:54,089 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:54,089 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:54,090 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:54,089 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B37A4A0444944283B5C65E2C19FC35BA, ordinal: 0
2016-05-17 20:45:54,090 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B37A4A0444944283B5C65E2C19FC35BA, mem on start (mb), free: 416, total: 553, max: 672
2016-05-17 20:45:54,090 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:54,090 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:54,091 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:54,091 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:54,091 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:54,091 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:54,091 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:54,091 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 20:45:54,092 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:54,092 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 20:45:54,092 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, EA413A1D8AA244E1B9EF6B102662BC59, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:54,092 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FEBA4EBF8BDB40D78748DD504B2E84DE, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 20:45:54,092 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:54,092 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: EA413A1D8AA244E1B9EF6B102662BC59 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:54,092 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:54,093 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:54,090 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:54,093 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:54,093 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:54,093 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 20:45:54,093 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:54,093 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 20:45:54,094 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 6837B65D2EA7478B8D296BC295D4907B, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:54,094 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FEBA4EBF8BDB40D78748DD504B2E84DE, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 20:45:54,094 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:54,094 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 6837B65D2EA7478B8D296BC295D4907B
2016-05-17 20:45:54,094 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:54,094 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 6837B65D2EA7478B8D296BC295D4907B
2016-05-17 20:45:54,094 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 6837B65D2EA7478B8D296BC295D4907B being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:54,095 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: EA413A1D8AA244E1B9EF6B102662BC59
2016-05-17 20:45:54,095 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:54,095 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:54,095 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:54,095 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:54,095 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: EA413A1D8AA244E1B9EF6B102662BC59
2016-05-17 20:45:54,096 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:54,096 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517953773_0001_1_00_000000_0
2016-05-17 20:45:54,097 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:54,098 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@169cc168
2016-05-17 20:45:54,099 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@169cc168
2016-05-17 20:45:54,102 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:54,102 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:54,102 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517953773_0001_1_01_000000_0
2016-05-17 20:45:54,101 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 20:45:54,103 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"] streamed: true, id: 6837B65D2EA7478B8D296BC295D4907B
2016-05-17 20:45:54,104 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]], id: A4FADDD631CF4E2793C0FF65D4349180
2016-05-17 20:45:54,104 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] A4FADDD631CF4E2793C0FF65D4349180
2016-05-17 20:45:54,106 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-05-17 20:45:54,107 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt"] streamed: true, id: EA413A1D8AA244E1B9EF6B102662BC59
2016-05-17 20:45:54,107 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]], id: A4FADDD631CF4E2793C0FF65D4349180
2016-05-17 20:45:54,107 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] A4FADDD631CF4E2793C0FF65D4349180
2016-05-17 20:45:54,110 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@27c90d11; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@78eb7f; comparator=cascading.tuple.hadoop.util.TupleComparator@1cdd50ca; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 20:45:54,110 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@2acdf17; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@236a6060; comparator=cascading.tuple.hadoop.util.TupleComparator@5e30c94; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 20:45:54,114 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 20:45:54,114 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 20:45:54,114 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:54,115 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 20:45:54,115 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 20:45:54,115 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:54,113 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:54,117 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@b2e359a
2016-05-17 20:45:54,118 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@b2e359a
2016-05-17 20:45:54,118 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:54,118 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:54,131 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 20:45:54,131 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 20:45:54,132 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 20:45:54,132 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 20:45:54,132 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 20:45:54,131 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 20:45:54,133 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 20:45:54,133 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 20:45:54,133 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 20:45:54,133 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-05-17 20:45:54,134 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:54,134 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F7C9F8290A98443ABC86347D3CDCB5E3, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:54,134 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 20:45:54,136 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517953773_0001_1_00_000000_0
2016-05-17 20:45:54,137 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F7C9F8290A98443ABC86347D3CDCB5E3, mem on close (mb), free: 213, total: 553, max: 672
2016-05-17 20:45:54,137 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 20:45:54,137 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 20:45:54,137 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-05-17 20:45:54,137 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-05-17 20:45:54,138 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-05-17 20:45:54,138 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:54,138 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 20:45:54,139 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 20:45:54,139 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B37A4A0444944283B5C65E2C19FC35BA, all 1 inputs ready in: 00:00:00.001
2016-05-17 20:45:54,142 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517953773_0001_1_01_000000_0
2016-05-17 20:45:54,143 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B37A4A0444944283B5C65E2C19FC35BA, mem on close (mb), free: 213, total: 553, max: 672
2016-05-17 20:45:54,143 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 20:45:54,144 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 20:45:54,144 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-05-17 20:45:54,144 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-05-17 20:45:54,491 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 20:45:54,491 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 20:45:54 Completed running task attempt: attempt_1463517953773_0001_1_00_000000_0
2016-05-17 20:45:54,495 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517953773_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:54 Completed running task attempt: attempt_1463517953773_0001_1_01_000000_0
2016-05-17 20:45:54,497 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517953773_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:54,498 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=1290, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=358, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463517954134, Process_Duration=2, Process_End_Time=1463517954136, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-17 20:45:54,498 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517953773_0001_1_00, eventCount=1
2016-05-17 20:45:54,499 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:54,499 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:54,499 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517953773_0001_1_01, eventCount=1
2016-05-17 20:45:54,499 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517953773_0001_00_000001
2016-05-17 20:45:54,499 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:54,500 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F7C9F8290A98443ABC86347D3CDCB5E3, taskAttemptId=attempt_1463517953773_0001_1_00_000000_0, startTime=1463517954025, finishTime=1463517954499, timeTaken=474, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=1290, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=358, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517954134, Process_Duration=2, Process_End_Time=1463517954136, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 20:45:54,500 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953773_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:54,501 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B37A4A0444944283B5C65E2C19FC35BA, taskAttemptId=attempt_1463517953773_0001_1_01_000000_0, startTime=1463517954060, finishTime=1463517954500, timeTaken=440, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=1290, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=343, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517954139, Process_Duration=3, Process_End_Time=1463517954142, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 20:45:54,501 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953773_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:54,501 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517953773_0001_1_00_000000_0
2016-05-17 20:45:54,501 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:TASK_FINISHED]: vertexName=F7C9F8290A98443ABC86347D3CDCB5E3, taskId=task_1463517953773_0001_1_00_000000, startTime=1463517954025, finishTime=1463517954501, timeTaken=476, status=SUCCEEDED, successfulAttemptID=attempt_1463517953773_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=1290, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=358, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517954134, Process_Duration=2, Process_End_Time=1463517954136, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 20:45:54,501 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953773_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:54,501 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:54,502 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:54,502 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:54,502 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517953773_0001_1_01_000000_0
2016-05-17 20:45:54,504 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:54,504 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=1290, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=343, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463517954139, Process_Duration=3, Process_End_Time=1463517954142, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-17 20:45:54,504 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:TASK_FINISHED]: vertexName=B37A4A0444944283B5C65E2C19FC35BA, taskId=task_1463517953773_0001_1_01_000000, startTime=1463517954060, finishTime=1463517954503, timeTaken=443, status=SUCCEEDED, successfulAttemptID=attempt_1463517953773_0001_1_01_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=1290, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=343, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517954139, Process_Duration=3, Process_End_Time=1463517954142, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-17 20:45:54,504 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:54,504 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:54,504 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:54,504 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953773_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:54,505 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3] : 1
2016-05-17 20:45:54,505 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3]
2016-05-17 20:45:54,505 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517953773_0001_00_000002
2016-05-17 20:45:54,506 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:54,507 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:VERTEX_FINISHED]: vertexName=F7C9F8290A98443ABC86347D3CDCB5E3, vertexId=vertex_1463517953773_0001_1_00, initRequestedTime=1463517953950, initedTime=1463517953970, startRequestedTime=1463517953972, startedTime=1463517953972, finishTime=1463517954505, timeTaken=533, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=1290, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=358, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517954134, Process_Duration=2, Process_End_Time=1463517954136, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463517954025, firstTasksToStart=[ task_1463517953773_0001_1_00_000000 ], lastTaskFinishTime=1463517954499, lastTasksToFinish=[ task_1463517953773_0001_1_00_000000 ], minTaskDuration=474, maxTaskDuration=474, avgTaskDuration=474.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517953773_0001_1_00_000000 ], longestDurationTasks=[ task_1463517953773_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:54,507 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:54,507 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA] : 1
2016-05-17 20:45:54,507 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA]
2016-05-17 20:45:54,509 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:54,509 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:VERTEX_FINISHED]: vertexName=B37A4A0444944283B5C65E2C19FC35BA, vertexId=vertex_1463517953773_0001_1_01, initRequestedTime=1463517953951, initedTime=1463517953978, startRequestedTime=1463517953982, startedTime=1463517953982, finishTime=1463517954507, timeTaken=525, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=1290, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=343, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517954139, Process_Duration=3, Process_End_Time=1463517954142, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463517954060, firstTasksToStart=[ task_1463517953773_0001_1_01_000000 ], lastTaskFinishTime=1463517954500, lastTasksToFinish=[ task_1463517953773_0001_1_01_000000 ], minTaskDuration=440, maxTaskDuration=440, avgTaskDuration=440.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517953773_0001_1_01_000000 ], longestDurationTasks=[ task_1463517953773_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:54,509 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:54,509 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:54,509 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:54,510 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517953773_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:54,510 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517953773_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:54,510 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517953773_0001_00_000001, Exiting
2016-05-17 20:45:54,510 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517953773_0001_00_000001
2016-05-17 20:45:54,510 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517953773_0001_00_000001
2016-05-17 20:45:54,511 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517953773_0001_00_000001, stoppedTime=1463517954510, exitStatus=0
2016-05-17 20:45:54,512 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:54,512 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517953773_0001_00_000002, stoppedTime=1463517954512, exitStatus=0
2016-05-17 20:45:54,512 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:54,511 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:54,513 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:54,514 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517953773_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 20:45:54,516 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517953773_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:54,516 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517953773_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:54,514 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE] attempt: attempt_1463517953773_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 20:45:54,516 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517953773_0001_00_000002, Exiting
2016-05-17 20:45:54,517 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517953773_0001_00_000002
2016-05-17 20:45:54,517 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517953773_0001_00_000002
2016-05-17 20:45:54,517 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 20:45:54,518 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE] attempt: attempt_1463517953773_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 20:45:54,518 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: FEBA4EBF8BDB40D78748DD504B2E84DE
2016-05-17 20:45:54,518 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE]
2016-05-17 20:45:54,518 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 20:45:54,518 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517953773_0001_00_000001
2016-05-17 20:45:54,518 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463517953773_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-17 20:45:54,519 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:54,519 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517953773_0001_00_000001 completed successfully
2016-05-17 20:45:54,518 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:TASK_STARTED]: vertexName=FEBA4EBF8BDB40D78748DD504B2E84DE, taskId=task_1463517953773_0001_1_02_000000, scheduledTime=1463517954518, launchTime=1463517954518
2016-05-17 20:45:54,519 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517953773_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:54,519 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953773_0001_1_02_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:54,520 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517953773_0001_00_000002 completed successfully
2016-05-17 20:45:54,520 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE]
2016-05-17 20:45:54,521 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 20:45:54,521 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 20:45:54,521 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 20:45:54,521 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 20:45:54,521 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: FEBA4EBF8BDB40D78748DD504B2E84DE, VertexParallelism: 1, TaskAttemptID:attempt_1463517953773_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B37A4A0444944283B5C65E2C19FC35BA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=F7C9F8290A98443ABC86347D3CDCB5E3, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=763782EE94B642ECB31B619719C37B70, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:54,521 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953773_0001_1_02_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:54,522 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:54,522 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517953773_0001_00_000003
2016-05-17 20:45:54,522 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000003 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:54,524 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517953773_0001_00_000003
2016-05-17 20:45:54,525 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 20:45:54,527 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000003 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:54,527 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:54,528 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517953773_0001_00_000003, launchTime=1463517954527
2016-05-17 20:45:54,528 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517953773_0001_00_000003
2016-05-17 20:45:54,529 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517953773_0001_1_02_000000_0] to container: [container_1463517953773_0001_00_000003]
2016-05-17 20:45:54,529 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000003 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:54,530 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517953773_0001_1_02_000000_0] started. Is using containerId: [container_1463517953773_0001_00_000003] on NM: [127.0.0.1:0]
2016-05-17 20:45:54,530 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FEBA4EBF8BDB40D78748DD504B2E84DE, taskAttemptId=attempt_1463517953773_0001_1_02_000000_0, startTime=1463517954530, containerId=container_1463517953773_0001_00_000003, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517953773_0001_00_000003/teamcity, completedLogs=
2016-05-17 20:45:54,530 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:54,530 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953773_0001_1_02_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:54,531 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953773_0001_1_02_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:54,532 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517953773_0001_00_000003 given task: attempt_1463517953773_0001_1_02_000000_0
2016-05-17 20:45:54,532 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517953773_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517953773_0001_1_02_000000_0
2016-05-17 20:45:54 Starting to run new task attempt: attempt_1463517953773_0001_1_02_000000_0
2016-05-17 20:45:54,533 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517953773_0001_1_02_000000_0
2016-05-17 20:45:54,533 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:54,533 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:54,533 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:54,533 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:54,533 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:54,533 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:54,533 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: FEBA4EBF8BDB40D78748DD504B2E84DE, VertexParallelism: 1, TaskAttemptID:attempt_1463517953773_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B37A4A0444944283B5C65E2C19FC35BA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=F7C9F8290A98443ABC86347D3CDCB5E3, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=763782EE94B642ECB31B619719C37B70, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:54,534 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:54,535 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517953773_0001_1_02_000000_0
2016-05-17 20:45:54,539 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:54,539 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:54,540 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517953773_0001_1_02_000000_0 sent events: (0-2)
2016-05-17 20:45:54,540 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B37A4A0444944283B5C65E2C19FC35BA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 20:45:54,541 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=763782EE94B642ECB31B619719C37B70, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:54,541 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:54,540 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:54,540 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F7C9F8290A98443ABC86347D3CDCB5E3, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 20:45:54,541 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:54,542 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:54,542 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 763782EE94B642ECB31B619719C37B70
2016-05-17 20:45:54,542 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:54,542 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 763782EE94B642ECB31B619719C37B70
2016-05-17 20:45:54,542 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F7C9F8290A98443ABC86347D3CDCB5E3
2016-05-17 20:45:54,549 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B37A4A0444944283B5C65E2C19FC35BA
2016-05-17 20:45:54,549 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:54,554 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 20:45:54,554 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 20:45:54,554 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B37A4A0444944283B5C65E2C19FC35BA
2016-05-17 20:45:54,556 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B37A4A0444944283B5C65E2C19FC35BA
2016-05-17 20:45:54,558 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 20:45:54,558 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 20:45:54,558 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: F7C9F8290A98443ABC86347D3CDCB5E3
2016-05-17 20:45:54,561 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F7C9F8290A98443ABC86347D3CDCB5E3
2016-05-17 20:45:54,561 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FEBA4EBF8BDB40D78748DD504B2E84DE, ordinal: 2
2016-05-17 20:45:54,561 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:54,562 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 763782EE94B642ECB31B619719C37B70
2016-05-17 20:45:54,562 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FEBA4EBF8BDB40D78748DD504B2E84DE, mem on start (mb), free: 424, total: 652, max: 672
2016-05-17 20:45:54,562 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:54,563 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 3 initializers to finish
2016-05-17 20:45:54,563 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:54,563 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:54,563 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:54,564 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:54,564 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:54,564 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:54,564 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 20:45:54,565 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 20:45:54,565 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345
2016-05-17 20:45:54,565 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 1268357504, TotalRequestedScaled: 6.0881160192E8, TotalJVMHeap: 704643072, TotalAvailable: 461541212, TotalRequested/TotalJVMHeap:1.80
2016-05-17 20:45:54,566 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 763782EE94B642ECB31B619719C37B70, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:54,566 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B37A4A0444944283B5C65E2C19FC35BA, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-05-17 20:45:54,566 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F7C9F8290A98443ABC86347D3CDCB5E3, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-05-17 20:45:54,566 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:54,567 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B37A4A0444944283B5C65E2C19FC35BA
2016-05-17 20:45:54,567 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B37A4A0444944283B5C65E2C19FC35BA being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:54,567 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:54,568 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 20:45:54,568 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B37A4A0444944283B5C65E2C19FC35BA with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 20:45:54,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F7C9F8290A98443ABC86347D3CDCB5E3
2016-05-17 20:45:54,568 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-05-17 20:45:54,569 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 20:45:54,570 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 20:45:54,569 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:54,569 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F7C9F8290A98443ABC86347D3CDCB5E3 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:54,570 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 20:45:54,573 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F7C9F8290A98443ABC86347D3CDCB5E3 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 20:45:54,573 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-05-17 20:45:54,574 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B37A4A0444944283B5C65E2C19FC35BA
2016-05-17 20:45:54,575 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 20:45:54,576 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 20:45:54,575 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 2
2016-05-17 20:45:54,578 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 2 IOs to start
2016-05-17 20:45:54,578 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:54,578 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F7C9F8290A98443ABC86347D3CDCB5E3
2016-05-17 20:45:54,578 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:54,579 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517953773_0001_1_02_000000_0
2016-05-17 20:45:54,580 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463517953773_0001_1_00_000000_0_10055, runDuration: 429323, ]
2016-05-17 20:45:54,581 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463517953773_0001_1_00_000000_0_10055] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1463517954581, TimeTaken=1, Rate=0.09 MB/s
2016-05-17 20:45:54,581 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F7C9F8290A98443ABC86347D3CDCB5E3
2016-05-17 20:45:54,581 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 20:45:54,581 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [F7C9F8290A98443ABC86347D3CDCB5E3] #10 in 1ms
2016-05-17 20:45:54,582 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F7C9F8290A98443ABC86347D3CDCB5E3] #10, status:false, isInterrupted:false
2016-05-17 20:45:54,583 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 20:45:54,584 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-05-17 20:45:54,584 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:54,585 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 20:45:54,585 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-17 20:45:54,585 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F7C9F8290A98443ABC86347D3CDCB5E3
2016-05-17 20:45:54,585 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]]
2016-05-17 20:45:54,585 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463517953773_0001_1_01_000000_0_10058, runDuration: 417847, ]
2016-05-17 20:45:54,585 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 20:45:54,586 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] streamed: true, id: A4FADDD631CF4E2793C0FF65D4349180
2016-05-17 20:45:54,586 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup"], id: 763782EE94B642ECB31B619719C37B70
2016-05-17 20:45:54,586 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup"]
2016-05-17 20:45:54,586 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] A4FADDD631CF4E2793C0FF65D4349180, for 2 inputs
2016-05-17 20:45:54,587 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] A4FADDD631CF4E2793C0FF65D4349180, for 2 inputs
2016-05-17 20:45:54,587 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463517953773_0001_1_01_000000_0_10058] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1463517954587, TimeTaken=1, Rate=0.09 MB/s
2016-05-17 20:45:54,587 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B37A4A0444944283B5C65E2C19FC35BA
2016-05-17 20:45:54,587 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 20:45:54,588 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [B37A4A0444944283B5C65E2C19FC35BA] #9 in 2ms
2016-05-17 20:45:54,590 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(89)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-17 20:45:54,590 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(94)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-17 20:45:54,593 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B37A4A0444944283B5C65E2C19FC35BA] #9, status:false, isInterrupted:false
2016-05-17 20:45:54,594 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 20:45:54,595 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-05-17 20:45:54,595 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:54,595 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 20:45:54,596 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-17 20:45:54,596 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FEBA4EBF8BDB40D78748DD504B2E84DE, all 2 inputs ready in: 00:00:00.006
2016-05-17 20:45:54,599 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@7a0d63a1; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@5081da1
2016-05-17 20:45:54,599 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B37A4A0444944283B5C65E2C19FC35BA
2016-05-17 20:45:54,600 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 20:45:54,602 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@57d1379b; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@6899ce27
2016-05-17 20:45:54,604 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:54,605 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:54,606 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517953773_0001_1_02_000000_0
2016-05-17 20:45:54,606 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517953773_0001_1_02_000000_0 given a go for committing the task output.
2016-05-17 20:45:54,607 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517953773_0001_1_02_000000_0
2016-05-17 20:45:54,608 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FEBA4EBF8BDB40D78748DD504B2E84DE, mem on close (mb), free: 420, total: 652, max: 672
2016-05-17 20:45:54,608 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B37A4A0444944283B5C65E2C19FC35BA
2016-05-17 20:45:54,609 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 20:45:54,610 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 20:45:54,610 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:54,611 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 20:45:54,611 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 20:45:54,612 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F7C9F8290A98443ABC86347D3CDCB5E3
2016-05-17 20:45:54,612 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 20:45:54,613 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 20:45:54,613 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:54,614 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 20:45:54,614 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 20:45:54 Completed running task attempt: attempt_1463517953773_0001_1_02_000000_0
2016-05-17 20:45:54,616 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517953773_0001_1_02_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:54,618 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=23, MERGE_PHASE_TIME=28, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=25][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463517954596, Process_Duration=11, Process_End_Time=1463517954607, Read_Duration=0, Tuples_Read=10, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-05-17 20:45:54,618 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:54,618 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FEBA4EBF8BDB40D78748DD504B2E84DE, taskAttemptId=attempt_1463517953773_0001_1_02_000000_0, startTime=1463517954530, finishTime=1463517954618, timeTaken=88, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=23, MERGE_PHASE_TIME=28, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=25, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517954596, Process_Duration=11, Process_End_Time=1463517954607, Read_Duration=0, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-17 20:45:54,619 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517953773_0001_1_02_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:54,619 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517953773_0001_1_02_000000_0
2016-05-17 20:45:54,620 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:54,619 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:54,620 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:54,620 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:TASK_FINISHED]: vertexName=FEBA4EBF8BDB40D78748DD504B2E84DE, taskId=task_1463517953773_0001_1_02_000000, startTime=1463517954530, finishTime=1463517954620, timeTaken=90, status=SUCCEEDED, successfulAttemptID=attempt_1463517953773_0001_1_02_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=23, MERGE_PHASE_TIME=28, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=25, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517954596, Process_Duration=11, Process_End_Time=1463517954607, Read_Duration=0, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-17 20:45:54,621 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517953773_0001_1_02_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:54,621 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE] : 1
2016-05-17 20:45:54,621 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE]
2016-05-17 20:45:54,622 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:54,622 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:VERTEX_FINISHED]: vertexName=FEBA4EBF8BDB40D78748DD504B2E84DE, vertexId=vertex_1463517953773_0001_1_02, initRequestedTime=1463517953955, initedTime=1463517953965, startRequestedTime=1463517953989, startedTime=1463517953989, finishTime=1463517954621, timeTaken=632, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=23, MERGE_PHASE_TIME=28, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=25, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517954596, Process_Duration=11, Process_End_Time=1463517954607, Read_Duration=0, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1463517954530, firstTasksToStart=[ task_1463517953773_0001_1_02_000000 ], lastTaskFinishTime=1463517954618, lastTasksToFinish=[ task_1463517953773_0001_1_02_000000 ], minTaskDuration=88, maxTaskDuration=88, avgTaskDuration=88.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517953773_0001_1_02_000000 ], longestDurationTasks=[ task_1463517953773_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:54,623 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:54,623 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-17 20:45:54,623 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517953773_0001_1
2016-05-17 20:45:54,623 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517953773_0001_1
2016-05-17 20:45:54,623 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 763782EE94B642ECB31B619719C37B70 for vertex: vertex_1463517953773_0001_1_02 [FEBA4EBF8BDB40D78748DD504B2E84DE]
2016-05-17 20:45:54,619 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:54,629 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517953773_0001_00_000003
2016-05-17 20:45:54,634 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463517953773_0001_1_00 [F7C9F8290A98443ABC86347D3CDCB5E3]
2016-05-17 20:45:54,634 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463517953773_0001_1_01 [B37A4A0444944283B5C65E2C19FC35BA]
2016-05-17 20:45:54,637 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517953773_0001_1, queueSize=0
2016-05-17 20:45:54,637 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:54,638 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517953773_0001_1, startTime=1463517953949, finishTime=1463517954634, timeTaken=685, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=2, File System Counters, FILE_BYTES_READ=502, FILE_BYTES_WRITTEN=2632, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=701, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=2052587520, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=15, OUTPUT_BYTES=150, OUTPUT_BYTES_WITH_OVERHEAD=182, OUTPUT_BYTES_PHYSICAL=190, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=23, MERGE_PHASE_TIME=28, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=25, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4390553862869, Process_Duration=16, Process_End_Time=4390553862885, Read_Duration=5, Tuples_Read=20, Tuples_Written=15, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=5
2016-05-17 20:45:54,638 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517953773_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:54,638 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517953773_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:54,639 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000003 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:54,642 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517953773_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517953773_0001_00_000003, stoppedTime=1463517954642, exitStatus=0
2016-05-17 20:45:54,642 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:54,642 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000003 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:54 Completed Dag: dag_1463517953773_0001_1
2016-05-17 20:45:54,643 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517953773_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:54,644 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517953773_0001_1_post
2016-05-17 20:45:54,644 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:54,644 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:54,644 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463517953773_0001_00_000003. Relying on regular task shutdown for it to end
2016-05-17 20:45:54,645 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517953773_0001_00_000003 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 20:45:54,645 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...estCoGroup/complexcogroup, applicationId=application_1463517953773_0001
2016-05-17 20:45:54,647 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:54,647 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:54,647 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:54,647 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:54,647 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:54,648 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:54,648 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:54,648 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:54,648 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:54,649 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:54,649 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:54,649 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:54,649 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:54,649 INFO task.TezChild (TezChild.java:run(198)) - Interrupted while waiting for new work for container container_1463517953773_0001_00_000003
2016-05-17 20:45:54,649 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517953773_0001_00_000003
2016-05-17 20:45:54,649 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517953773_0001_00_000003
2016-05-17 20:45:54,655 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B2C08DB7C11E4969A6E4DD97BC95997E/B8BC7CD64C0149BD9DDFBAEB86C03A24/.tez/application_1463517953773_0001
2016-05-17 20:45:54,656 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup/_temporary
2016-05-17 20:45:54,656 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] completed in: 00:00.902
2016-05-17 20:45:54,729 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup
2016-05-17 20:45:54,751 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:54,751 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:54,786 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 20:45:54,790 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-17 20:45:54,790 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 20:45:54,790 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 20:45:54,790 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:54,804 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 20:45:54,804 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace
2016-05-17 20:45:54,812 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 20:45:54,812 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:54,813 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace"]
2016-05-17 20:45:54,813 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 20:45:54,814 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 20:45:54,814 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 20:45:54,815 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...tform/testReplace/replace
2016-05-17 20:45:54,815 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:54,827 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 20:45:54,831 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:54,832 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:54,832 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:54,841 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7366C99430DE4DD1BABA4485D4C0CDF3/E2D9B6A116C54DF8A59557216AC46905/.tez/application_1463517954832_0001 doesn't exist and is created
2016-05-17 20:45:54,855 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:54,855 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:54,856 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7366C99430DE4DD1BABA4485D4C0CDF3/E2D9B6A116C54DF8A59557216AC46905/.tez/application_1463517954832_0001
2016-05-17 20:45:54,858 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517954832_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:54,869 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:54,869 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:54,871 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:54,871 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:54,871 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7366C99430DE4DD1BABA4485D4C0CDF3/E2D9B6A116C54DF8A59557216AC46905/.tez/application_1463517954832_0001/localmode-log-dir
2016-05-17 20:45:54,871 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7366C99430DE4DD1BABA4485D4C0CDF3/E2D9B6A116C54DF8A59557216AC46905/.tez/application_1463517954832_0001/localmode-log-dir/history.txt.appattempt_1463517954832_0001_000000, maxErrors=10
2016-05-17 20:45:54,871 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:54,872 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517954832_0001_000000, appSubmitTime=1463517954858, launchTime=1463517954858
2016-05-17 20:45:54,874 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:54,876 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:54,878 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:54,878 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:54,880 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:43863
2016-05-17 20:45:54,881 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:54,884 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517954832_0001_000000, startTime=1463517954884
2016-05-17 20:45:54,884 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:54,884 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:54,955 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:54,956 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:54,956 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...tform/testReplace/replace, applicationId=application_1463517954832_0001, dagName=[7366C99430DE4DD1BABA4485D4C0CDF3/E2D9B6A116C54DF8A59557216AC46905] test/(1/1) ...tform/testReplace/replace
2016-05-17 20:45:54,979 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [7366C99430DE4DD1BABA4485D4C0CDF3/E2D9B6A116C54DF8A59557216AC46905] test/(1/1) ...tform/testReplace/replace
2016-05-17 20:45:54,980 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517954832_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7366C99430DE4DD1BABA4485D4C0CDF3/E2D9B6A116C54DF8A59557216AC46905/.tez/application_1463517954832_0001/localmode-log-dir/dag_1463517954832_0001_1.dot
2016-05-17 20:45:54,981 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7366C99430DE4DD1BABA4485D4C0CDF3/E2D9B6A116C54DF8A59557216AC46905/.tez/application_1463517954832_0001/localmode-log-dir/dag_1463517954832_0001_1-tez-dag.pb.txt
2016-05-17 20:45:54,989 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517954832_0001_1
2016-05-17 20:45:54,989 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:54,990 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [7366C99430DE4DD1BABA4485D4C0CDF3/E2D9B6A116C54DF8A59557216AC46905] test/(1/1) ...tform/testReplace/replace
2016-05-17 20:45:54 Running Dag: dag_1463517954832_0001_1
2016-05-17 20:45:55,002 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517954832_0001_1, submitTime=1463517954979
2016-05-17 20:45:55,003 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:55,003 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E80C71241AB144E3807B712750E7D22B
2016-05-17 20:45:55,003 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:55,004 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:55,005 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517954832_0001_1, initTime=1463517955003
2016-05-17 20:45:55,005 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517954832_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:55,005 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:DAG_STARTED]: dagID=dag_1463517954832_0001_1, startTime=1463517955005
2016-05-17 20:45:55,005 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:55,005 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:55,006 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517954832_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:55,007 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E80C71241AB144E3807B712750E7D22B : {6DBAD42B0AD64641B5A2DC3C90FB0E50={InputName=6DBAD42B0AD64641B5A2DC3C90FB0E50}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:55,007 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 6DBAD42B0AD64641B5A2DC3C90FB0E50, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:55,008 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B]
2016-05-17 20:45:55,008 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...tform/testReplace/replace, applicationId=application_1463517954832_0001, dagName=[7366C99430DE4DD1BABA4485D4C0CDF3/E2D9B6A116C54DF8A59557216AC46905] test/(1/1) ...tform/testReplace/replace
2016-05-17 20:45:55,008 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B]
2016-05-17 20:45:55,008 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B]
2016-05-17 20:45:55,008 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B]. Starting root input initializers: 1
2016-05-17 20:45:55,011 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:55,011 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 6DBAD42B0AD64641B5A2DC3C90FB0E50 on vertex vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B]
2016-05-17 20:45:55,013 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:55,013 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 6DBAD42B0AD64641B5A2DC3C90FB0E50 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:55,013 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:55,014 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:55,015 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 20:45:55,016 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 6DBAD42B0AD64641B5A2DC3C90FB0E50 on vertex vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B]
2016-05-17 20:45:55,016 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:55,016 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {6DBAD42B0AD64641B5A2DC3C90FB0E50=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:55,016 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B] parallelism set to 1
2016-05-17 20:45:55,017 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:55,017 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517954832_0001_1_00, eventCount=1
2016-05-17 20:45:55,017 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B]
2016-05-17 20:45:55,017 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=4AE0D34413814F0E9E38CAB108E48018, vertexId=vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:55,017 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=4AE0D34413814F0E9E38CAB108E48018, vertexId=vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B]
2016-05-17 20:45:55,019 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=4AE0D34413814F0E9E38CAB108E48018, vertexName=E80C71241AB144E3807B712750E7D22B, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:55,019 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=4AE0D34413814F0E9E38CAB108E48018, vertexId=vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B]
2016-05-17 20:45:55,021 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E80C71241AB144E3807B712750E7D22B, vertexId=vertex_1463517954832_0001_1_00, initRequestedTime=1463517955007, initedTime=1463517955021, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:55,021 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B] with distanceFromRoot: 0
2016-05-17 20:45:55,021 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:55,022 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:55,022 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E80C71241AB144E3807B712750E7D22B
2016-05-17 20:45:55,023 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B]
2016-05-17 20:45:55,023 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:55,024 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517954832_0001_1_00, startRequestedTime=1463517955022, startedTime=1463517955022
2016-05-17 20:45:55,024 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:55,025 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:55,025 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:TASK_STARTED]: vertexName=E80C71241AB144E3807B712750E7D22B, taskId=task_1463517954832_0001_1_00_000000, scheduledTime=1463517955025, launchTime=1463517955025
2016-05-17 20:45:55,026 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517954832_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:55,026 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B]
2016-05-17 20:45:55,026 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463517954832_0001, with dag id: dag_1463517954832_0001_1
2016-05-17 20:45:55,028 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [7366C99430DE4DD1BABA4485D4C0CDF3/E2D9B6A116C54DF8A59557216AC46905] test/(1/1) ...tform/testReplace/replace, VertexName: E80C71241AB144E3807B712750E7D22B, VertexParallelism: 1, TaskAttemptID:attempt_1463517954832_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=6DBAD42B0AD64641B5A2DC3C90FB0E50, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=4AE0D34413814F0E9E38CAB108E48018, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:55,029 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517954832_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:55,030 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:55,033 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:55,033 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:55,033 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:55,034 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517954832_0001_00_000001
2016-05-17 20:45:55,034 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517954832_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:55,036 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517954832_0001_00_000001
2016-05-17 20:45:55,036 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:55,039 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517954832_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:55,039 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:55,039 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517954832_0001_00_000001, launchTime=1463517955039
2016-05-17 20:45:55,041 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517954832_0001_00_000001
2016-05-17 20:45:55,042 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517954832_0001_1_00_000000_0] to container: [container_1463517954832_0001_00_000001]
2016-05-17 20:45:55,042 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517954832_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:55,042 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517954832_0001_00_000001 given task: attempt_1463517954832_0001_1_00_000000_0
2016-05-17 20:45:55,043 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517954832_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517954832_0001_1_00_000000_0
2016-05-17 20:45:55,043 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517954832_0001_1_00_000000_0] started. Is using containerId: [container_1463517954832_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:55,043 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E80C71241AB144E3807B712750E7D22B, taskAttemptId=attempt_1463517954832_0001_1_00_000000_0, startTime=1463517955042, containerId=container_1463517954832_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517954832_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:55,043 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517954832_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:55,044 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517954832_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:55,044 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:55 Starting to run new task attempt: attempt_1463517954832_0001_1_00_000000_0
2016-05-17 20:45:55,045 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517954832_0001_1_00_000000_0
2016-05-17 20:45:55,045 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:55,046 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:55,046 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:55,046 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:55,047 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:55,047 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:55,048 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7366C99430DE4DD1BABA4485D4C0CDF3/E2D9B6A116C54DF8A59557216AC46905] test/(1/1) ...tform/testReplace/replace, VertexName: E80C71241AB144E3807B712750E7D22B, VertexParallelism: 1, TaskAttemptID:attempt_1463517954832_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=6DBAD42B0AD64641B5A2DC3C90FB0E50, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=4AE0D34413814F0E9E38CAB108E48018, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:55,048 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:55,049 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517954832_0001_1_00_000000_0
2016-05-17 20:45:55,049 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:55,049 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:55,049 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=6DBAD42B0AD64641B5A2DC3C90FB0E50, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:55,050 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:55,050 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 6DBAD42B0AD64641B5A2DC3C90FB0E50
2016-05-17 20:45:55,050 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 6DBAD42B0AD64641B5A2DC3C90FB0E50
2016-05-17 20:45:55,052 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:55,053 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=4AE0D34413814F0E9E38CAB108E48018, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:55,053 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:55,053 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 4AE0D34413814F0E9E38CAB108E48018
2016-05-17 20:45:55,053 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:55,054 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 4AE0D34413814F0E9E38CAB108E48018
2016-05-17 20:45:55,052 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517954832_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:55,056 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:55,055 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:55,059 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 6DBAD42B0AD64641B5A2DC3C90FB0E50
2016-05-17 20:45:55,060 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 6DBAD42B0AD64641B5A2DC3C90FB0E50
2016-05-17 20:45:55,066 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E80C71241AB144E3807B712750E7D22B, ordinal: 0
2016-05-17 20:45:55,066 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E80C71241AB144E3807B712750E7D22B, mem on start (mb), free: 348, total: 652, max: 672
2016-05-17 20:45:55,066 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:55,067 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:55,067 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:55,069 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:55,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 4AE0D34413814F0E9E38CAB108E48018
2016-05-17 20:45:55,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:55,070 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:55,070 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:55,070 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:55,071 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:55,071 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:55,071 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:55,072 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 20:45:55,072 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 6DBAD42B0AD64641B5A2DC3C90FB0E50, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:55,072 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 4AE0D34413814F0E9E38CAB108E48018, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:55,073 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:55,073 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 6DBAD42B0AD64641B5A2DC3C90FB0E50
2016-05-17 20:45:55,073 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:55,073 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 6DBAD42B0AD64641B5A2DC3C90FB0E50
2016-05-17 20:45:55,074 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 6DBAD42B0AD64641B5A2DC3C90FB0E50 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:55,074 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:55,075 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:55,075 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:55,076 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517954832_0001_1_00_000000_0
2016-05-17 20:45:55,076 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:55,077 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1ee9e51b
2016-05-17 20:45:55,078 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1ee9e51b
2016-05-17 20:45:55,078 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:55,078 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:55,079 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:55,080 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 6DBAD42B0AD64641B5A2DC3C90FB0E50
2016-05-17 20:45:55,080 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace"], id: 4AE0D34413814F0E9E38CAB108E48018
2016-05-17 20:45:55,081 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace"]
2016-05-17 20:45:55,081 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:55,081 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:55,082 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E80C71241AB144E3807B712750E7D22B, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:55,083 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:55,084 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:55,084 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517954832_0001_1_00_000000_0
2016-05-17 20:45:55,084 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517954832_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 20:45:55,085 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517954832_0001_1_00_000000_0
2016-05-17 20:45:55,085 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E80C71241AB144E3807B712750E7D22B, mem on close (mb), free: 346, total: 652, max: 672
2016-05-17 20:45:55 Completed running task attempt: attempt_1463517954832_0001_1_00_000000_0
2016-05-17 20:45:55,086 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517954832_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:55,089 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10][cascading.flow.SliceCounters Process_Begin_Time=1463517955082, Process_Duration=3, Process_End_Time=1463517955085, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10, Tuples_Written=10]]
2016-05-17 20:45:55,089 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:55,089 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:55,089 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:55,089 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E80C71241AB144E3807B712750E7D22B, taskAttemptId=attempt_1463517954832_0001_1_00_000000_0, startTime=1463517955042, finishTime=1463517955089, timeTaken=47, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463517955082, Process_Duration=3, Process_End_Time=1463517955085, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-17 20:45:55,089 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517954832_0001_00_000001
2016-05-17 20:45:55,090 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517954832_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:55,090 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:55,090 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517954832_0001_1_00_000000_0
2016-05-17 20:45:55,091 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:55,091 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:TASK_FINISHED]: vertexName=E80C71241AB144E3807B712750E7D22B, taskId=task_1463517954832_0001_1_00_000000, startTime=1463517955042, finishTime=1463517955091, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1463517954832_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463517955082, Process_Duration=3, Process_End_Time=1463517955085, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-17 20:45:55,091 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517954832_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:55,092 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B] : 1
2016-05-17 20:45:55,092 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B]
2016-05-17 20:45:55,093 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:55,093 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:VERTEX_FINISHED]: vertexName=E80C71241AB144E3807B712750E7D22B, vertexId=vertex_1463517954832_0001_1_00, initRequestedTime=1463517955007, initedTime=1463517955021, startRequestedTime=1463517955022, startedTime=1463517955022, finishTime=1463517955092, timeTaken=70, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463517955082, Process_Duration=3, Process_End_Time=1463517955085, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10, vertexStats=firstTaskStartTime=1463517955042, firstTasksToStart=[ task_1463517954832_0001_1_00_000000 ], lastTaskFinishTime=1463517955089, lastTasksToFinish=[ task_1463517954832_0001_1_00_000000 ], minTaskDuration=47, maxTaskDuration=47, avgTaskDuration=47.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517954832_0001_1_00_000000 ], longestDurationTasks=[ task_1463517954832_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:55,093 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:55,094 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517954832_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:55,094 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517954832_0001_00_000001, stoppedTime=1463517955094, exitStatus=0
2016-05-17 20:45:55,094 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:55,094 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517954832_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:55,094 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 20:45:55,094 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517954832_0001_1
2016-05-17 20:45:55,094 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517954832_0001_1
2016-05-17 20:45:55,095 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 4AE0D34413814F0E9E38CAB108E48018 for vertex: vertex_1463517954832_0001_1_00 [E80C71241AB144E3807B712750E7D22B]
2016-05-17 20:45:55,101 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517954832_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:55,101 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517954832_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:55,101 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517954832_0001_00_000001, Exiting
2016-05-17 20:45:55,101 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517954832_0001_00_000001
2016-05-17 20:45:55,101 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517954832_0001_00_000001
2016-05-17 20:45:55,103 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517954832_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:55,103 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517954832_0001_00_000001 completed successfully
2016-05-17 20:45:55,105 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517954832_0001_1, queueSize=0
2016-05-17 20:45:55,105 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:55,106 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517954832_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517954832_0001_1, startTime=1463517955005, finishTime=1463517955103, timeTaken=98, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684195840, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463517955082, Process_Duration=3, Process_End_Time=1463517955085, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-17 20:45:55,106 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517954832_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:55,106 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517954832_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:55,107 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517954832_0001_00_000001
2016-05-17 20:45:55,107 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517954832_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:55 Completed Dag: dag_1463517954832_0001_1
2016-05-17 20:45:55,109 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517954832_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:55,110 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517954832_0001_1_post
2016-05-17 20:45:55,110 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:55,111 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:55,113 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...tform/testReplace/replace, applicationId=application_1463517954832_0001
2016-05-17 20:45:55,115 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:55,115 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:55,115 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:55,115 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:55,115 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:55,116 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:55,116 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:55,116 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:55,117 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:55,117 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:55,117 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:55,118 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:55,118 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:55,122 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7366C99430DE4DD1BABA4485D4C0CDF3/E2D9B6A116C54DF8A59557216AC46905/.tez/application_1463517954832_0001
2016-05-17 20:45:55,122 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace/_temporary
2016-05-17 20:45:55,123 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.310
2016-05-17 20:45:55,155 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testReplace/replace
2016-05-17 20:45:55,188 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:55,189 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:55,224 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 20:45:55,236 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.023
2016-05-17 20:45:55,237 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:55,237 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:55,237 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:55,255 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-05-17 20:45:55,261 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2
2016-05-17 20:45:55,291 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 20:45:55,292 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 20:45:55,306 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting
2016-05-17 20:45:55,307 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 20:45:55,307 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2"]
2016-05-17 20:45:55,308 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] parallel execution of steps is enabled: false
2016-05-17 20:45:55,308 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executing total steps: 1
2016-05-17 20:45:55,308 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] allocating management threads: 1
2016-05-17 20:45:55,310 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting step: (1/1) ...latform/testCount2/count2
2016-05-17 20:45:55,311 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:55,322 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tez session mode enabled: true
2016-05-17 20:45:55,327 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:55,327 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:55,328 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:55,338 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA/.tez/application_1463517955327_0001 doesn't exist and is created
2016-05-17 20:45:55,356 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:55,356 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:55,357 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA/.tez/application_1463517955327_0001
2016-05-17 20:45:55,360 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517955327_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:55,370 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:55,371 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:55,372 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:55,373 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:55,373 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA/.tez/application_1463517955327_0001/localmode-log-dir
2016-05-17 20:45:55,373 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA/.tez/application_1463517955327_0001/localmode-log-dir/history.txt.appattempt_1463517955327_0001_000000, maxErrors=10
2016-05-17 20:45:55,373 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:55,374 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517955327_0001_000000, appSubmitTime=1463517955359, launchTime=1463517955359
2016-05-17 20:45:55,376 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:55,377 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:55,379 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:55,379 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:55,385 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:35668
2016-05-17 20:45:55,387 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:55,389 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517955327_0001_000000, startTime=1463517955389
2016-05-17 20:45:55,389 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:55,390 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:55,456 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:55,457 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:55,457 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testCount2/count2, applicationId=application_1463517955327_0001, dagName=[C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA] count/(1/1) ...latform/testCount2/count2
2016-05-17 20:45:55,475 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA] count/(1/1) ...latform/testCount2/count2
2016-05-17 20:45:55,477 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517955327_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA/.tez/application_1463517955327_0001/localmode-log-dir/dag_1463517955327_0001_1.dot
2016-05-17 20:45:55,477 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA/.tez/application_1463517955327_0001/localmode-log-dir/dag_1463517955327_0001_1-tez-dag.pb.txt
2016-05-17 20:45:55,485 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517955327_0001_1
2016-05-17 20:45:55,485 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:55,485 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA] count/(1/1) ...latform/testCount2/count2
2016-05-17 20:45:55 Running Dag: dag_1463517955327_0001_1
2016-05-17 20:45:55,496 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517955327_0001_1, submitTime=1463517955475
2016-05-17 20:45:55,496 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:55,497 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DD516F6A13164CD2BF1515BC64D72E2A
2016-05-17 20:45:55,497 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:55,497 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517955327_0001_1, initTime=1463517955496
2016-05-17 20:45:55,497 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517955327_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:55,497 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:55,498 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:DAG_STARTED]: dagID=dag_1463517955327_0001_1, startTime=1463517955498
2016-05-17 20:45:55,498 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:55,498 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:55,498 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517955327_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:55,499 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D4C620047B804607BE524C3D6E8A1D1E : {8959902C54374501A5283B13E8C43754={InputName=8959902C54374501A5283B13E8C43754}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:55,499 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 8959902C54374501A5283B13E8C43754, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:55,499 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E]
2016-05-17 20:45:55,499 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testCount2/count2, applicationId=application_1463517955327_0001, dagName=[C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA] count/(1/1) ...latform/testCount2/count2
2016-05-17 20:45:55,499 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E]
2016-05-17 20:45:55,499 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E]
2016-05-17 20:45:55,500 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E]. Starting root input initializers: 1
2016-05-17 20:45:55,500 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:55,500 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A]
2016-05-17 20:45:55,501 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 8959902C54374501A5283B13E8C43754 on vertex vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E]
2016-05-17 20:45:55,503 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:55,503 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 20:45:55,504 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A]
2016-05-17 20:45:55,504 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A]
2016-05-17 20:45:55,504 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A]
2016-05-17 20:45:55,505 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=7448873726D14BBB9B8229B6DCBDB440, vertexId=vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:55,505 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=7448873726D14BBB9B8229B6DCBDB440, vertexId=vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A]
2016-05-17 20:45:55,504 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 8959902C54374501A5283B13E8C43754 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:55,506 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:55,506 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:55,507 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=7448873726D14BBB9B8229B6DCBDB440, vertexName=DD516F6A13164CD2BF1515BC64D72E2A, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:55,508 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=7448873726D14BBB9B8229B6DCBDB440, vertexId=vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A]
2016-05-17 20:45:55,508 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-05-17 20:45:55,509 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 8959902C54374501A5283B13E8C43754 on vertex vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E]
2016-05-17 20:45:55,509 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:55,510 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DD516F6A13164CD2BF1515BC64D72E2A, vertexId=vertex_1463517955327_0001_1_01, initRequestedTime=1463517955500, initedTime=1463517955509, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 20:45:55,510 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A] transitioned from NEW to INITED due to event V_INIT
2016-05-17 20:45:55,510 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:55,510 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {8959902C54374501A5283B13E8C43754=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:55,511 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E] parallelism set to 1
2016-05-17 20:45:55,511 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:55,511 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517955327_0001_1_00, eventCount=1
2016-05-17 20:45:55,512 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:55,512 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D4C620047B804607BE524C3D6E8A1D1E, vertexId=vertex_1463517955327_0001_1_00, initRequestedTime=1463517955499, initedTime=1463517955512, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:55,512 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E] with distanceFromRoot: 0
2016-05-17 20:45:55,513 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:55,513 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D4C620047B804607BE524C3D6E8A1D1E
2016-05-17 20:45:55,513 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E]
2016-05-17 20:45:55,514 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:55,514 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517955327_0001_1_00, startRequestedTime=1463517955513, startedTime=1463517955513
2016-05-17 20:45:55,515 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:55,515 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:55,516 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:TASK_STARTED]: vertexName=D4C620047B804607BE524C3D6E8A1D1E, taskId=task_1463517955327_0001_1_00_000000, scheduledTime=1463517955515, launchTime=1463517955515
2016-05-17 20:45:55,516 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517955327_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:55,516 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463517955327_0001_1_00 for vertex: vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A] numStartedSources: 1 numSources: 1
2016-05-17 20:45:55,516 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A] with distanceFromRoot: 1
2016-05-17 20:45:55,517 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E]
2016-05-17 20:45:55,517 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D4C620047B804607BE524C3D6E8A1D1E in vertex: DD516F6A13164CD2BF1515BC64D72E2A
2016-05-17 20:45:55,517 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: DD516F6A13164CD2BF1515BC64D72E2A with 1 source tasks and 1 pending tasks
2016-05-17 20:45:55,518 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:55,518 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517955327_0001_1_01, startRequestedTime=1463517955517, startedTime=1463517955517
2016-05-17 20:45:55,518 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:55,519 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA] count/(1/1) ...latform/testCount2/count2, VertexName: D4C620047B804607BE524C3D6E8A1D1E, VertexParallelism: 1, TaskAttemptID:attempt_1463517955327_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8959902C54374501A5283B13E8C43754, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DD516F6A13164CD2BF1515BC64D72E2A, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:55,519 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517955327_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:55,519 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] submitted tez dag to app master: application_1463517955327_0001, with dag id: dag_1463517955327_0001_1
2016-05-17 20:45:55,520 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:55,523 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:55,524 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:55,524 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:55,525 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517955327_0001_00_000001
2016-05-17 20:45:55,525 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955327_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:55,528 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517955327_0001_00_000001
2016-05-17 20:45:55,528 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:55,529 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955327_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:55,529 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:55,529 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517955327_0001_00_000001, launchTime=1463517955529
2016-05-17 20:45:55,530 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517955327_0001_00_000001
2016-05-17 20:45:55,532 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517955327_0001_1_00_000000_0] to container: [container_1463517955327_0001_00_000001]
2016-05-17 20:45:55,532 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955327_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:55,532 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517955327_0001_00_000001 given task: attempt_1463517955327_0001_1_00_000000_0
2016-05-17 20:45:55,532 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517955327_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517955327_0001_1_00_000000_0
2016-05-17 20:45:55,532 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517955327_0001_1_00_000000_0] started. Is using containerId: [container_1463517955327_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:55,533 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D4C620047B804607BE524C3D6E8A1D1E, taskAttemptId=attempt_1463517955327_0001_1_00_000000_0, startTime=1463517955532, containerId=container_1463517955327_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517955327_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:55,533 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517955327_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:55,533 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:55,533 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517955327_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:55 Starting to run new task attempt: attempt_1463517955327_0001_1_00_000000_0
2016-05-17 20:45:55,534 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517955327_0001_1_00_000000_0
2016-05-17 20:45:55,534 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:55,534 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:55,534 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:55,535 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:55,535 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:55,536 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:55,536 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA] count/(1/1) ...latform/testCount2/count2, VertexName: D4C620047B804607BE524C3D6E8A1D1E, VertexParallelism: 1, TaskAttemptID:attempt_1463517955327_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8959902C54374501A5283B13E8C43754, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DD516F6A13164CD2BF1515BC64D72E2A, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:55,536 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:55,536 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517955327_0001_1_00_000000_0
2016-05-17 20:45:55,537 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:55,537 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:55,537 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517955327_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:55,538 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:55,538 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=8959902C54374501A5283B13E8C43754, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:55,540 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:55,543 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DD516F6A13164CD2BF1515BC64D72E2A, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 20:45:55,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:55,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DD516F6A13164CD2BF1515BC64D72E2A
2016-05-17 20:45:55,546 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:55,546 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 8959902C54374501A5283B13E8C43754
2016-05-17 20:45:55,546 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 8959902C54374501A5283B13E8C43754
2016-05-17 20:45:55,547 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 20:45:55,547 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: DD516F6A13164CD2BF1515BC64D72E2A
2016-05-17 20:45:55,552 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DD516F6A13164CD2BF1515BC64D72E2A
2016-05-17 20:45:55,552 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D4C620047B804607BE524C3D6E8A1D1E, ordinal: 0
2016-05-17 20:45:55,552 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D4C620047B804607BE524C3D6E8A1D1E, mem on start (mb), free: 266, total: 652, max: 672
2016-05-17 20:45:55,553 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:55,553 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:55,553 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:55,553 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:55,553 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 8959902C54374501A5283B13E8C43754
2016-05-17 20:45:55,553 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 8959902C54374501A5283B13E8C43754
2016-05-17 20:45:55,554 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:55,554 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:55,554 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:55,555 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:55,555 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 20:45:55,555 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:55,555 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 20:45:55,556 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 8959902C54374501A5283B13E8C43754, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:55,556 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DD516F6A13164CD2BF1515BC64D72E2A, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 20:45:55,556 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:55,556 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 8959902C54374501A5283B13E8C43754 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:55,556 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 8959902C54374501A5283B13E8C43754
2016-05-17 20:45:55,557 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:55,557 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 8959902C54374501A5283B13E8C43754
2016-05-17 20:45:55,557 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:55,558 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:55,558 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:55,559 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517955327_0001_1_00_000000_0
2016-05-17 20:45:55,562 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 20:45:55,562 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: 8959902C54374501A5283B13E8C43754
2016-05-17 20:45:55,562 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: E801256111E8442FA8FA072793C4C3B5
2016-05-17 20:45:55,563 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] E801256111E8442FA8FA072793C4C3B5
2016-05-17 20:45:55,565 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@2111bc5f; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@136611ef; comparator=cascading.tuple.hadoop.util.TupleComparator@12236bdc; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 20:45:55,565 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 20:45:55,566 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 20:45:55,566 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:55,579 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:55,579 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@37dd7c4e
2016-05-17 20:45:55,580 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@37dd7c4e
2016-05-17 20:45:55,581 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:55,581 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:55,591 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 20:45:55,592 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 20:45:55,592 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 20:45:55,592 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 20:45:55,592 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 20:45:55,594 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 20:45:55,594 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:55,595 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D4C620047B804607BE524C3D6E8A1D1E, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:55,602 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517955327_0001_1_00_000000_0
2016-05-17 20:45:55,602 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D4C620047B804607BE524C3D6E8A1D1E, mem on close (mb), free: 323, total: 653, max: 672
2016-05-17 20:45:55,602 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 20:45:55,603 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 20:45:55,603 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2016-05-17 20:45:55,603 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 20:45:55,607 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 20:45:55 Completed running task attempt: attempt_1463517955327_0001_1_00_000000_0
2016-05-17 20:45:55,611 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517955327_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:55,612 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517955327_0001_1_00, eventCount=1
2016-05-17 20:45:55,612 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:55,612 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D4C620047B804607BE524C3D6E8A1D1E, taskAttemptId=attempt_1463517955327_0001_1_00_000000_0, startTime=1463517955532, finishTime=1463517955612, timeTaken=80, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684720128, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517955595, Process_Duration=7, Process_End_Time=1463517955602, Read_Duration=5, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 20:45:55,612 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517955327_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:55,613 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517955327_0001_1_00_000000_0
2016-05-17 20:45:55,613 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:55,613 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:TASK_FINISHED]: vertexName=D4C620047B804607BE524C3D6E8A1D1E, taskId=task_1463517955327_0001_1_00_000000, startTime=1463517955532, finishTime=1463517955613, timeTaken=81, status=SUCCEEDED, successfulAttemptID=attempt_1463517955327_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684720128, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517955595, Process_Duration=7, Process_End_Time=1463517955602, Read_Duration=5, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 20:45:55,613 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:55,614 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517955327_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:55,614 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E] : 1
2016-05-17 20:45:55,614 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E]
2016-05-17 20:45:55,615 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:55,615 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:VERTEX_FINISHED]: vertexName=D4C620047B804607BE524C3D6E8A1D1E, vertexId=vertex_1463517955327_0001_1_00, initRequestedTime=1463517955499, initedTime=1463517955512, startRequestedTime=1463517955513, startedTime=1463517955513, finishTime=1463517955614, timeTaken=101, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684720128, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517955595, Process_Duration=7, Process_End_Time=1463517955602, Read_Duration=5, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463517955532, firstTasksToStart=[ task_1463517955327_0001_1_00_000000 ], lastTaskFinishTime=1463517955612, lastTasksToFinish=[ task_1463517955327_0001_1_00_000000 ], minTaskDuration=80, maxTaskDuration=80, avgTaskDuration=80.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517955327_0001_1_00_000000 ], longestDurationTasks=[ task_1463517955327_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:55,616 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:55,616 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955327_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:55,616 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517955327_0001_00_000001, stoppedTime=1463517955616, exitStatus=0
2016-05-17 20:45:55,616 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:55,616 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955327_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:55,616 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A] attempt: attempt_1463517955327_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 20:45:55,617 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: DD516F6A13164CD2BF1515BC64D72E2A
2016-05-17 20:45:55,617 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A]
2016-05-17 20:45:55,617 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:55,618 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463517955327_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 20:45:55,618 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:TASK_STARTED]: vertexName=DD516F6A13164CD2BF1515BC64D72E2A, taskId=task_1463517955327_0001_1_01_000000, scheduledTime=1463517955618, launchTime=1463517955618
2016-05-17 20:45:55,619 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517955327_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:55,619 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955327_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 20:45:55,619 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A]
2016-05-17 20:45:55,620 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA] count/(1/1) ...latform/testCount2/count2, VertexName: DD516F6A13164CD2BF1515BC64D72E2A, VertexParallelism: 1, TaskAttemptID:attempt_1463517955327_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D4C620047B804607BE524C3D6E8A1D1E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=7448873726D14BBB9B8229B6DCBDB440, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:55,618 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:55,621 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517955327_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:55,621 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:55,622 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517955327_0001_00_000002
2016-05-17 20:45:55,622 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955327_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:55,624 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517955327_0001_00_000002
2016-05-17 20:45:55,624 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:55,625 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:55,625 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517955327_0001_00_000002, launchTime=1463517955624
2016-05-17 20:45:55,625 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684720128, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463517955595, Process_Duration=7, Process_End_Time=1463517955602, Read_Duration=5, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 20:45:55,625 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955327_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:55,626 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517955327_0001_00_000002
2016-05-17 20:45:55,626 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:55,626 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:55,626 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517955327_0001_1_01_000000_0] to container: [container_1463517955327_0001_00_000002]
2016-05-17 20:45:55,627 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955327_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:55,627 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517955327_0001_1_01_000000_0] started. Is using containerId: [container_1463517955327_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 20:45:55,627 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DD516F6A13164CD2BF1515BC64D72E2A, taskAttemptId=attempt_1463517955327_0001_1_01_000000_0, startTime=1463517955627, containerId=container_1463517955327_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517955327_0001_00_000002/teamcity, completedLogs=
2016-05-17 20:45:55,627 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:55,628 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517955327_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:55,626 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517955327_0001_00_000001
2016-05-17 20:45:55,628 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517955327_0001_00_000002 given task: attempt_1463517955327_0001_1_01_000000_0
2016-05-17 20:45:55,628 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517955327_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:55,628 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517955327_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:55,628 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517955327_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:55,628 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517955327_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517955327_0001_1_01_000000_0
2016-05-17 20:45:55,628 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517955327_0001_00_000001, Exiting
2016-05-17 20:45:55,629 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517955327_0001_00_000001
2016-05-17 20:45:55,629 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517955327_0001_00_000001
2016-05-17 20:45:55 Starting to run new task attempt: attempt_1463517955327_0001_1_01_000000_0
2016-05-17 20:45:55,630 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517955327_0001_1_01_000000_0
2016-05-17 20:45:55,630 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:55,630 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:55,630 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:55,630 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:55,631 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:55,631 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:55,631 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA] count/(1/1) ...latform/testCount2/count2, VertexName: DD516F6A13164CD2BF1515BC64D72E2A, VertexParallelism: 1, TaskAttemptID:attempt_1463517955327_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D4C620047B804607BE524C3D6E8A1D1E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=7448873726D14BBB9B8229B6DCBDB440, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:55,631 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:55,631 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517955327_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:55,631 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517955327_0001_00_000001 completed successfully
2016-05-17 20:45:55,632 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955327_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 20:45:55,632 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517955327_0001_1_01_000000_0
2016-05-17 20:45:55,634 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:55,634 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:55,634 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517955327_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 20:45:55,636 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:55,638 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:55,641 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=7448873726D14BBB9B8229B6DCBDB440, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:55,642 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D4C620047B804607BE524C3D6E8A1D1E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 20:45:55,642 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:55,642 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:55,642 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D4C620047B804607BE524C3D6E8A1D1E
2016-05-17 20:45:55,642 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 7448873726D14BBB9B8229B6DCBDB440
2016-05-17 20:45:55,643 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:55,644 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 7448873726D14BBB9B8229B6DCBDB440
2016-05-17 20:45:55,644 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 20:45:55,645 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 20:45:55,645 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D4C620047B804607BE524C3D6E8A1D1E
2016-05-17 20:45:55,644 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DD516F6A13164CD2BF1515BC64D72E2A, ordinal: 1
2016-05-17 20:45:55,646 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DD516F6A13164CD2BF1515BC64D72E2A, mem on start (mb), free: 318, total: 653, max: 672
2016-05-17 20:45:55,646 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:55,648 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:55,646 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D4C620047B804607BE524C3D6E8A1D1E
2016-05-17 20:45:55,648 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:55,654 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:55,654 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 7448873726D14BBB9B8229B6DCBDB440
2016-05-17 20:45:55,655 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:55,655 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:55,656 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:55,656 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:55,656 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 20:45:55,657 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:55,658 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 20:45:55,658 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 7448873726D14BBB9B8229B6DCBDB440, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:55,658 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D4C620047B804607BE524C3D6E8A1D1E, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 20:45:55,658 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:55,658 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D4C620047B804607BE524C3D6E8A1D1E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:55,659 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:55,659 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:55,659 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D4C620047B804607BE524C3D6E8A1D1E
2016-05-17 20:45:55,660 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:55,661 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 20:45:55,662 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D4C620047B804607BE524C3D6E8A1D1E with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 20:45:55,662 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 20:45:55,662 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 20:45:55,662 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 20:45:55,663 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D4C620047B804607BE524C3D6E8A1D1E
2016-05-17 20:45:55,663 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:55,663 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517955327_0001_1_01_000000_0
2016-05-17 20:45:55,667 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463517955327_0001_1_00_000000_0_10067, runDuration: 64495, ]
2016-05-17 20:45:55,667 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-05-17 20:45:55,668 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: E801256111E8442FA8FA072793C4C3B5
2016-05-17 20:45:55,668 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2"], id: 7448873726D14BBB9B8229B6DCBDB440
2016-05-17 20:45:55,669 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2"]
2016-05-17 20:45:55,669 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] E801256111E8442FA8FA072793C4C3B5, for 1 inputs
2016-05-17 20:45:55,669 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463517955327_0001_1_00_000000_0_10067] to DISK_DIRECT, CompressedSize=463, DecompressedSize=459,EndTime=1463517955669, TimeTaken=1, Rate=0.44 MB/s
2016-05-17 20:45:55,669 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D4C620047B804607BE524C3D6E8A1D1E
2016-05-17 20:45:55,670 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 20:45:55,670 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [D4C620047B804607BE524C3D6E8A1D1E] #11 in 2ms
2016-05-17 20:45:55,670 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D4C620047B804607BE524C3D6E8A1D1E] #11, status:false, isInterrupted:false
2016-05-17 20:45:55,671 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 20:45:55,671 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 463 bytes from disk
2016-05-17 20:45:55,671 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:55,672 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 20:45:55,672 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-05-17 20:45:55,672 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D4C620047B804607BE524C3D6E8A1D1E
2016-05-17 20:45:55,672 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 20:45:55,673 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DD516F6A13164CD2BF1515BC64D72E2A, all 1 inputs ready in: 00:00:00.004
2016-05-17 20:45:55,677 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@317db632; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@3444a600
2016-05-17 20:45:55,680 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:55,680 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:55,680 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517955327_0001_1_01_000000_0
2016-05-17 20:45:55,681 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517955327_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 20:45:55,681 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517955327_0001_1_01_000000_0
2016-05-17 20:45:55,681 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DD516F6A13164CD2BF1515BC64D72E2A, mem on close (mb), free: 315, total: 653, max: 672
2016-05-17 20:45:55,682 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D4C620047B804607BE524C3D6E8A1D1E
2016-05-17 20:45:55,682 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 20:45:55,683 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 20:45:55,683 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:55,683 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 20:45:55,683 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 20:45:55 Completed running task attempt: attempt_1463517955327_0001_1_01_000000_0
2016-05-17 20:45:55,684 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517955327_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:55,685 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:55,685 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DD516F6A13164CD2BF1515BC64D72E2A, taskAttemptId=attempt_1463517955327_0001_1_01_000000_0, startTime=1463517955627, finishTime=1463517955685, timeTaken=58, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684720128, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517955674, Process_Duration=7, Process_End_Time=1463517955681, Read_Duration=0, Tuples_Read=20, Tuples_Written=17, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 20:45:55,686 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517955327_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:55,686 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517955327_0001_1_01_000000_0
2016-05-17 20:45:55,686 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:55,686 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:55,687 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:TASK_FINISHED]: vertexName=DD516F6A13164CD2BF1515BC64D72E2A, taskId=task_1463517955327_0001_1_01_000000, startTime=1463517955627, finishTime=1463517955686, timeTaken=59, status=SUCCEEDED, successfulAttemptID=attempt_1463517955327_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684720128, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517955674, Process_Duration=7, Process_End_Time=1463517955681, Read_Duration=0, Tuples_Read=20, Tuples_Written=17, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 20:45:55,687 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517955327_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:55,687 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A] : 1
2016-05-17 20:45:55,687 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A]
2016-05-17 20:45:55,688 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:55,688 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:VERTEX_FINISHED]: vertexName=DD516F6A13164CD2BF1515BC64D72E2A, vertexId=vertex_1463517955327_0001_1_01, initRequestedTime=1463517955500, initedTime=1463517955509, startRequestedTime=1463517955517, startedTime=1463517955517, finishTime=1463517955687, timeTaken=170, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684720128, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517955674, Process_Duration=7, Process_End_Time=1463517955681, Read_Duration=0, Tuples_Read=20, Tuples_Written=17, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1463517955627, firstTasksToStart=[ task_1463517955327_0001_1_01_000000 ], lastTaskFinishTime=1463517955685, lastTasksToFinish=[ task_1463517955327_0001_1_01_000000 ], minTaskDuration=58, maxTaskDuration=58, avgTaskDuration=58.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517955327_0001_1_01_000000 ], longestDurationTasks=[ task_1463517955327_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:55,689 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:55,689 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955327_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:55,689 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517955327_0001_00_000002, stoppedTime=1463517955689, exitStatus=0
2016-05-17 20:45:55,689 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:55,690 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955327_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:55,690 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:55,690 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517955327_0001_1
2016-05-17 20:45:55,690 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517955327_0001_1
2016-05-17 20:45:55,691 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 7448873726D14BBB9B8229B6DCBDB440 for vertex: vertex_1463517955327_0001_1_01 [DD516F6A13164CD2BF1515BC64D72E2A]
2016-05-17 20:45:55,690 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=684720128, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463517955674, Process_Duration=7, Process_End_Time=1463517955681, Read_Duration=0, Tuples_Read=20, Tuples_Written=17, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=17]]
2016-05-17 20:45:55,692 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:55,692 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:55,692 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517955327_0001_00_000002
2016-05-17 20:45:55,693 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517955327_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 20:45:55,693 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517955327_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:55,693 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517955327_0001_00_000002, Exiting
2016-05-17 20:45:55,693 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517955327_0001_00_000002
2016-05-17 20:45:55,693 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517955327_0001_00_000002
2016-05-17 20:45:55,694 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517955327_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:55,694 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517955327_0001_00_000002 completed successfully
2016-05-17 20:45:55,699 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463517955327_0001_1_00 [D4C620047B804607BE524C3D6E8A1D1E]
2016-05-17 20:45:55,702 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517955327_0001_1, queueSize=0
2016-05-17 20:45:55,702 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:55,704 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955327_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517955327_0001_1, startTime=1463517955498, finishTime=1463517955699, timeTaken=201, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=940, FILE_BYTES_WRITTEN=777, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1369440256, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927035911269, Process_Duration=14, Process_End_Time=2927035911283, Read_Duration=5, Tuples_Read=40, Tuples_Written=37, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-05-17 20:45:55,704 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517955327_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:55,705 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517955327_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:55,707 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517955327_0001_00_000002
2016-05-17 20:45:55,708 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955327_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:55 Completed Dag: dag_1463517955327_0001_1
2016-05-17 20:45:55,710 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517955327_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:55,711 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517955327_0001_1_post
2016-05-17 20:45:55,711 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:55,712 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:55,714 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testCount2/count2, applicationId=application_1463517955327_0001
2016-05-17 20:45:55,718 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:55,718 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:55,718 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:55,718 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:55,718 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:55,719 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:55,719 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:55,720 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:55,720 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:55,720 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:55,720 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:55,720 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:55,719 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:55,726 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C92E508D27EA489A87A91EFAEA5C82E2/ED8B020CDAFF4A15882E2FA49BE524FA/.tez/application_1463517955327_0001
2016-05-17 20:45:55,726 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2/_temporary
2016-05-17 20:45:55,727 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] completed in: 00:00.420
2016-05-17 20:45:55,799 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount2/count2
2016-05-17 20:45:55,829 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:55,829 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:55,863 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 20:45:55,882 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.029
2016-05-17 20:45:55,884 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:55,884 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:55,884 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:55,905 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-05-17 20:45:55,911 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3
2016-05-17 20:45:55,955 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting
2016-05-17 20:45:55,955 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 20:45:55,955 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3"]
2016-05-17 20:45:55,956 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] parallel execution of steps is enabled: false
2016-05-17 20:45:55,956 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executing total steps: 1
2016-05-17 20:45:55,956 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] allocating management threads: 1
2016-05-17 20:45:55,957 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting step: (1/1) ...latform/testCount3/count3
2016-05-17 20:45:55,957 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:55,968 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tez session mode enabled: true
2016-05-17 20:45:55,973 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:55,973 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:55,973 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:55,982 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A/.tez/application_1463517955973_0001 doesn't exist and is created
2016-05-17 20:45:55,997 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:55,998 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:55,998 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A/.tez/application_1463517955973_0001
2016-05-17 20:45:56,001 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517955973_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:56,011 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:56,012 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:56,013 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:56,014 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:56,014 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A/.tez/application_1463517955973_0001/localmode-log-dir
2016-05-17 20:45:56,014 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A/.tez/application_1463517955973_0001/localmode-log-dir/history.txt.appattempt_1463517955973_0001_000000, maxErrors=10
2016-05-17 20:45:56,015 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:56,015 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517955973_0001_000000, appSubmitTime=1463517956001, launchTime=1463517956001
2016-05-17 20:45:56,019 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:56,021 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:56,022 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:56,022 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:56,022 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:50231
2016-05-17 20:45:56,026 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:56,029 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517955973_0001_000000, startTime=1463517956029
2016-05-17 20:45:56,029 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:56,029 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:56,098 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:56,098 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:56,098 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testCount3/count3, applicationId=application_1463517955973_0001, dagName=[2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A] count/(1/1) ...latform/testCount3/count3
2016-05-17 20:45:56,114 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A] count/(1/1) ...latform/testCount3/count3
2016-05-17 20:45:56,116 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517955973_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A/.tez/application_1463517955973_0001/localmode-log-dir/dag_1463517955973_0001_1.dot
2016-05-17 20:45:56,117 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A/.tez/application_1463517955973_0001/localmode-log-dir/dag_1463517955973_0001_1-tez-dag.pb.txt
2016-05-17 20:45:56,128 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517955973_0001_1
2016-05-17 20:45:56,129 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:56,129 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A] count/(1/1) ...latform/testCount3/count3
2016-05-17 20:45:56 Running Dag: dag_1463517955973_0001_1
2016-05-17 20:45:56,143 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517955973_0001_1, submitTime=1463517956115
2016-05-17 20:45:56,143 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:56,144 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A3F494A478E84615BDF9C31E63BA6EDC
2016-05-17 20:45:56,144 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:56,144 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517955973_0001_1, initTime=1463517956143
2016-05-17 20:45:56,144 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:56,145 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517955973_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:56,146 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:DAG_STARTED]: dagID=dag_1463517955973_0001_1, startTime=1463517956146
2016-05-17 20:45:56,146 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:56,146 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:56,146 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517955973_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:56,146 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E97316F327FA4D2D84F09F72A535DE7D : {082D7BE07AB1443F9FA3B873961105C2={InputName=082D7BE07AB1443F9FA3B873961105C2}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:56,147 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 082D7BE07AB1443F9FA3B873961105C2, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:56,147 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D]
2016-05-17 20:45:56,147 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D]
2016-05-17 20:45:56,147 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D]
2016-05-17 20:45:56,148 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testCount3/count3, applicationId=application_1463517955973_0001, dagName=[2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A] count/(1/1) ...latform/testCount3/count3
2016-05-17 20:45:56,148 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D]. Starting root input initializers: 1
2016-05-17 20:45:56,148 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:56,148 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC]
2016-05-17 20:45:56,150 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 082D7BE07AB1443F9FA3B873961105C2 on vertex vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D]
2016-05-17 20:45:56,151 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 20:45:56,151 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC]
2016-05-17 20:45:56,152 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC]
2016-05-17 20:45:56,152 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC]
2016-05-17 20:45:56,152 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=DA0740327CBD4EC1994A080E5DD4B3A3, vertexId=vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:56,152 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:56,153 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=DA0740327CBD4EC1994A080E5DD4B3A3, vertexId=vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC]
2016-05-17 20:45:56,153 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 082D7BE07AB1443F9FA3B873961105C2 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:56,154 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:56,154 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:56,155 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=DA0740327CBD4EC1994A080E5DD4B3A3, vertexName=A3F494A478E84615BDF9C31E63BA6EDC, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:56,155 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=DA0740327CBD4EC1994A080E5DD4B3A3, vertexId=vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC]
2016-05-17 20:45:56,157 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-05-17 20:45:56,157 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:56,157 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A3F494A478E84615BDF9C31E63BA6EDC, vertexId=vertex_1463517955973_0001_1_01, initRequestedTime=1463517956148, initedTime=1463517956157, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 20:45:56,157 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC] transitioned from NEW to INITED due to event V_INIT
2016-05-17 20:45:56,158 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 082D7BE07AB1443F9FA3B873961105C2 on vertex vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D]
2016-05-17 20:45:56,158 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:56,159 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {082D7BE07AB1443F9FA3B873961105C2=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:56,159 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D] parallelism set to 1
2016-05-17 20:45:56,160 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:56,160 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517955973_0001_1_00, eventCount=1
2016-05-17 20:45:56,161 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E97316F327FA4D2D84F09F72A535DE7D, vertexId=vertex_1463517955973_0001_1_00, initRequestedTime=1463517956146, initedTime=1463517956161, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:56,161 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D] with distanceFromRoot: 0
2016-05-17 20:45:56,161 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:56,162 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:56,162 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E97316F327FA4D2D84F09F72A535DE7D
2016-05-17 20:45:56,162 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D]
2016-05-17 20:45:56,162 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:56,163 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517955973_0001_1_00, startRequestedTime=1463517956162, startedTime=1463517956162
2016-05-17 20:45:56,163 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:56,163 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:56,163 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:TASK_STARTED]: vertexName=E97316F327FA4D2D84F09F72A535DE7D, taskId=task_1463517955973_0001_1_00_000000, scheduledTime=1463517956163, launchTime=1463517956163
2016-05-17 20:45:56,164 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517955973_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:56,164 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463517955973_0001_1_00 for vertex: vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC] numStartedSources: 1 numSources: 1
2016-05-17 20:45:56,164 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC] with distanceFromRoot: 1
2016-05-17 20:45:56,164 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D]
2016-05-17 20:45:56,166 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E97316F327FA4D2D84F09F72A535DE7D in vertex: A3F494A478E84615BDF9C31E63BA6EDC
2016-05-17 20:45:56,166 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A3F494A478E84615BDF9C31E63BA6EDC with 1 source tasks and 1 pending tasks
2016-05-17 20:45:56,166 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] submitted tez dag to app master: application_1463517955973_0001, with dag id: dag_1463517955973_0001_1
2016-05-17 20:45:56,168 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:56,168 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517955973_0001_1_01, startRequestedTime=1463517956166, startedTime=1463517956166
2016-05-17 20:45:56,169 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:56,169 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A] count/(1/1) ...latform/testCount3/count3, VertexName: E97316F327FA4D2D84F09F72A535DE7D, VertexParallelism: 1, TaskAttemptID:attempt_1463517955973_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=082D7BE07AB1443F9FA3B873961105C2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A3F494A478E84615BDF9C31E63BA6EDC, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:56,170 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517955973_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:56,172 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:56,173 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:56,173 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:56,173 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:56,174 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517955973_0001_00_000001
2016-05-17 20:45:56,174 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955973_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:56,176 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517955973_0001_00_000001
2016-05-17 20:45:56,176 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:56,177 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955973_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:56,177 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:56,177 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517955973_0001_00_000001, launchTime=1463517956177
2016-05-17 20:45:56,178 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517955973_0001_00_000001
2016-05-17 20:45:56,178 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517955973_0001_1_00_000000_0] to container: [container_1463517955973_0001_00_000001]
2016-05-17 20:45:56,178 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955973_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:56,179 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517955973_0001_00_000001 given task: attempt_1463517955973_0001_1_00_000000_0
2016-05-17 20:45:56,179 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517955973_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517955973_0001_1_00_000000_0
2016-05-17 20:45:56,179 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517955973_0001_1_00_000000_0] started. Is using containerId: [container_1463517955973_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:56,179 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E97316F327FA4D2D84F09F72A535DE7D, taskAttemptId=attempt_1463517955973_0001_1_00_000000_0, startTime=1463517956179, containerId=container_1463517955973_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517955973_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:56,179 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:56,179 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517955973_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:56,179 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517955973_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:56 Starting to run new task attempt: attempt_1463517955973_0001_1_00_000000_0
2016-05-17 20:45:56,179 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517955973_0001_1_00_000000_0
2016-05-17 20:45:56,180 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:56,180 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:56,180 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:56,180 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:56,180 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:56,180 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:56,180 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A] count/(1/1) ...latform/testCount3/count3, VertexName: E97316F327FA4D2D84F09F72A535DE7D, VertexParallelism: 1, TaskAttemptID:attempt_1463517955973_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=082D7BE07AB1443F9FA3B873961105C2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A3F494A478E84615BDF9C31E63BA6EDC, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:56,181 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:56,181 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517955973_0001_1_00_000000_0
2016-05-17 20:45:56,181 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:56,181 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:56,182 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=082D7BE07AB1443F9FA3B873961105C2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:56,183 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:56,183 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 082D7BE07AB1443F9FA3B873961105C2
2016-05-17 20:45:56,183 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 082D7BE07AB1443F9FA3B873961105C2
2016-05-17 20:45:56,185 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:56,189 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517955973_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:56,190 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A3F494A478E84615BDF9C31E63BA6EDC, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 20:45:56,190 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:56,190 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A3F494A478E84615BDF9C31E63BA6EDC
2016-05-17 20:45:56,192 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:56,192 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:56,194 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 082D7BE07AB1443F9FA3B873961105C2
2016-05-17 20:45:56,195 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 082D7BE07AB1443F9FA3B873961105C2
2016-05-17 20:45:56,195 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 20:45:56,195 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 20:45:56,196 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 20:45:56,196 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A3F494A478E84615BDF9C31E63BA6EDC
2016-05-17 20:45:56,197 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A3F494A478E84615BDF9C31E63BA6EDC
2016-05-17 20:45:56,199 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E97316F327FA4D2D84F09F72A535DE7D, ordinal: 0
2016-05-17 20:45:56,199 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E97316F327FA4D2D84F09F72A535DE7D, mem on start (mb), free: 226, total: 653, max: 672
2016-05-17 20:45:56,199 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:56,199 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:56,199 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:56,199 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:56,199 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:56,200 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:56,200 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:56,200 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 20:45:56,200 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:56,200 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 20:45:56,201 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 082D7BE07AB1443F9FA3B873961105C2, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:56,201 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A3F494A478E84615BDF9C31E63BA6EDC, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 20:45:56,201 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:56,201 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 082D7BE07AB1443F9FA3B873961105C2 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:56,202 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:56,202 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:56,202 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 082D7BE07AB1443F9FA3B873961105C2
2016-05-17 20:45:56,202 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:56,203 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 082D7BE07AB1443F9FA3B873961105C2
2016-05-17 20:45:56,203 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:56,204 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517955973_0001_1_00_000000_0
2016-05-17 20:45:56,205 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:56,205 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6f40f6c5
2016-05-17 20:45:56,207 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6f40f6c5
2016-05-17 20:45:56,208 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:56,208 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:56,207 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 20:45:56,209 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: 082D7BE07AB1443F9FA3B873961105C2
2016-05-17 20:45:56,210 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: 31FFDA18ACE24D1D8BDD6EAFAA2226C0
2016-05-17 20:45:56,210 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] 31FFDA18ACE24D1D8BDD6EAFAA2226C0
2016-05-17 20:45:56,212 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@b9641a6; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@35afff3b; comparator=cascading.tuple.hadoop.util.TupleComparator@359d9606; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 20:45:56,213 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 20:45:56,213 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 20:45:56,214 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:56,238 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 20:45:56,239 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 20:45:56,239 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 20:45:56,239 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 20:45:56,240 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 20:45:56,241 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 20:45:56,241 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:56,241 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E97316F327FA4D2D84F09F72A535DE7D, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:56,244 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517955973_0001_1_00_000000_0
2016-05-17 20:45:56,245 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E97316F327FA4D2D84F09F72A535DE7D, mem on close (mb), free: 323, total: 653, max: 672
2016-05-17 20:45:56,245 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 20:45:56,245 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 20:45:56,245 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2016-05-17 20:45:56,246 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 20:45:56,250 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 20:45:56 Completed running task attempt: attempt_1463517955973_0001_1_00_000000_0
2016-05-17 20:45:56,254 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517955973_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:56,256 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517955973_0001_1_00, eventCount=1
2016-05-17 20:45:56,256 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:56,257 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E97316F327FA4D2D84F09F72A535DE7D, taskAttemptId=attempt_1463517955973_0001_1_00_000000_0, startTime=1463517956179, finishTime=1463517956256, timeTaken=77, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=685244416, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517956242, Process_Duration=2, Process_End_Time=1463517956244, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 20:45:56,257 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517955973_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:56,257 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517955973_0001_1_00_000000_0
2016-05-17 20:45:56,257 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:56,257 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:56,257 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:TASK_FINISHED]: vertexName=E97316F327FA4D2D84F09F72A535DE7D, taskId=task_1463517955973_0001_1_00_000000, startTime=1463517956179, finishTime=1463517956257, timeTaken=78, status=SUCCEEDED, successfulAttemptID=attempt_1463517955973_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=685244416, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517956242, Process_Duration=2, Process_End_Time=1463517956244, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 20:45:56,258 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517955973_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:56,258 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D] : 1
2016-05-17 20:45:56,258 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D]
2016-05-17 20:45:56,259 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:56,259 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:VERTEX_FINISHED]: vertexName=E97316F327FA4D2D84F09F72A535DE7D, vertexId=vertex_1463517955973_0001_1_00, initRequestedTime=1463517956146, initedTime=1463517956161, startRequestedTime=1463517956162, startedTime=1463517956162, finishTime=1463517956258, timeTaken=96, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=685244416, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517956242, Process_Duration=2, Process_End_Time=1463517956244, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463517956179, firstTasksToStart=[ task_1463517955973_0001_1_00_000000 ], lastTaskFinishTime=1463517956256, lastTasksToFinish=[ task_1463517955973_0001_1_00_000000 ], minTaskDuration=77, maxTaskDuration=77, avgTaskDuration=77.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517955973_0001_1_00_000000 ], longestDurationTasks=[ task_1463517955973_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:56,259 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:56,260 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955973_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:56,260 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC] attempt: attempt_1463517955973_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 20:45:56,260 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A3F494A478E84615BDF9C31E63BA6EDC
2016-05-17 20:45:56,260 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC]
2016-05-17 20:45:56,260 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517955973_0001_00_000001, stoppedTime=1463517956260, exitStatus=0
2016-05-17 20:45:56,261 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:56,261 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955973_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:56,261 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:56,261 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:TASK_STARTED]: vertexName=A3F494A478E84615BDF9C31E63BA6EDC, taskId=task_1463517955973_0001_1_01_000000, scheduledTime=1463517956261, launchTime=1463517956261
2016-05-17 20:45:56,261 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:56,261 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=685244416, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463517956242, Process_Duration=2, Process_End_Time=1463517956244, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 20:45:56,261 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517955973_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:56,262 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463517955973_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 20:45:56,262 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:56,262 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:56,262 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC]
2016-05-17 20:45:56,263 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955973_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 20:45:56,263 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A] count/(1/1) ...latform/testCount3/count3, VertexName: A3F494A478E84615BDF9C31E63BA6EDC, VertexParallelism: 1, TaskAttemptID:attempt_1463517955973_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E97316F327FA4D2D84F09F72A535DE7D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=DA0740327CBD4EC1994A080E5DD4B3A3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:56,263 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517955973_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:56,263 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:56,265 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517955973_0001_00_000002
2016-05-17 20:45:56,265 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955973_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:56,266 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517955973_0001_00_000001
2016-05-17 20:45:56,266 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517955973_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:56,266 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517955973_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:56,266 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517955973_0001_00_000001, Exiting
2016-05-17 20:45:56,266 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517955973_0001_00_000001
2016-05-17 20:45:56,266 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517955973_0001_00_000001
2016-05-17 20:45:56,268 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517955973_0001_00_000002
2016-05-17 20:45:56,268 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517955973_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:56,268 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517955973_0001_00_000001 completed successfully
2016-05-17 20:45:56,268 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 20:45:56,268 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955973_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 20:45:56,270 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955973_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:56,270 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:56,270 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517955973_0001_00_000002, launchTime=1463517956270
2016-05-17 20:45:56,271 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517955973_0001_00_000002
2016-05-17 20:45:56,272 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517955973_0001_1_01_000000_0] to container: [container_1463517955973_0001_00_000002]
2016-05-17 20:45:56,272 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955973_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:56,272 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517955973_0001_1_01_000000_0] started. Is using containerId: [container_1463517955973_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 20:45:56,272 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A3F494A478E84615BDF9C31E63BA6EDC, taskAttemptId=attempt_1463517955973_0001_1_01_000000_0, startTime=1463517956272, containerId=container_1463517955973_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517955973_0001_00_000002/teamcity, completedLogs=
2016-05-17 20:45:56,273 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:56,273 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517955973_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:56,273 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517955973_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:56,273 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517955973_0001_00_000002 given task: attempt_1463517955973_0001_1_01_000000_0
2016-05-17 20:45:56,273 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517955973_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517955973_0001_1_01_000000_0
2016-05-17 20:45:56 Starting to run new task attempt: attempt_1463517955973_0001_1_01_000000_0
2016-05-17 20:45:56,273 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517955973_0001_1_01_000000_0
2016-05-17 20:45:56,274 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:56,274 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:56,274 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:56,274 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:56,274 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:56,274 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:56,274 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A] count/(1/1) ...latform/testCount3/count3, VertexName: A3F494A478E84615BDF9C31E63BA6EDC, VertexParallelism: 1, TaskAttemptID:attempt_1463517955973_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E97316F327FA4D2D84F09F72A535DE7D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=DA0740327CBD4EC1994A080E5DD4B3A3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:56,275 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:56,275 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517955973_0001_1_01_000000_0
2016-05-17 20:45:56,275 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:56,275 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:56,276 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517955973_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 20:45:56,276 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:56,277 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E97316F327FA4D2D84F09F72A535DE7D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 20:45:56,277 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:56,278 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E97316F327FA4D2D84F09F72A535DE7D
2016-05-17 20:45:56,278 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DA0740327CBD4EC1994A080E5DD4B3A3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:56,278 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:56,279 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DA0740327CBD4EC1994A080E5DD4B3A3
2016-05-17 20:45:56,279 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:56,279 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: DA0740327CBD4EC1994A080E5DD4B3A3
2016-05-17 20:45:56,279 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:56,287 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 20:45:56,287 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 20:45:56,288 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: E97316F327FA4D2D84F09F72A535DE7D
2016-05-17 20:45:56,295 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:56,295 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DA0740327CBD4EC1994A080E5DD4B3A3
2016-05-17 20:45:56,295 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A3F494A478E84615BDF9C31E63BA6EDC, ordinal: 1
2016-05-17 20:45:56,296 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A3F494A478E84615BDF9C31E63BA6EDC, mem on start (mb), free: 317, total: 653, max: 672
2016-05-17 20:45:56,296 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:56,296 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:56,296 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:56,297 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E97316F327FA4D2D84F09F72A535DE7D
2016-05-17 20:45:56,297 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:56,297 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:56,298 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:56,298 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:56,298 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 20:45:56,298 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:56,298 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 20:45:56,298 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DA0740327CBD4EC1994A080E5DD4B3A3, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:56,299 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E97316F327FA4D2D84F09F72A535DE7D, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 20:45:56,299 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:56,299 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E97316F327FA4D2D84F09F72A535DE7D
2016-05-17 20:45:56,301 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:56,301 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 20:45:56,302 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E97316F327FA4D2D84F09F72A535DE7D with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 20:45:56,302 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 20:45:56,302 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 20:45:56,302 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 20:45:56,302 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E97316F327FA4D2D84F09F72A535DE7D being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:56,307 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E97316F327FA4D2D84F09F72A535DE7D
2016-05-17 20:45:56,307 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:56,307 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:56,307 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:56,307 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517955973_0001_1_01_000000_0
2016-05-17 20:45:56,308 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463517955973_0001_1_00_000000_0_10074, runDuration: 59609, ]
2016-05-17 20:45:56,310 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463517955973_0001_1_00_000000_0_10074] to DISK_DIRECT, CompressedSize=463, DecompressedSize=459,EndTime=1463517956310, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 20:45:56,310 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E97316F327FA4D2D84F09F72A535DE7D
2016-05-17 20:45:56,310 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-05-17 20:45:56,311 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 20:45:56,311 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [E97316F327FA4D2D84F09F72A535DE7D] #12 in 2ms
2016-05-17 20:45:56,312 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E97316F327FA4D2D84F09F72A535DE7D] #12, status:false, isInterrupted:false
2016-05-17 20:45:56,312 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: 31FFDA18ACE24D1D8BDD6EAFAA2226C0
2016-05-17 20:45:56,312 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3"], id: DA0740327CBD4EC1994A080E5DD4B3A3
2016-05-17 20:45:56,312 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3"]
2016-05-17 20:45:56,313 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] 31FFDA18ACE24D1D8BDD6EAFAA2226C0, for 1 inputs
2016-05-17 20:45:56,314 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 20:45:56,314 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 463 bytes from disk
2016-05-17 20:45:56,314 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:56,315 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 20:45:56,315 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-05-17 20:45:56,315 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E97316F327FA4D2D84F09F72A535DE7D
2016-05-17 20:45:56,315 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 20:45:56,316 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A3F494A478E84615BDF9C31E63BA6EDC, all 1 inputs ready in: 00:00:00.003
2016-05-17 20:45:56,318 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@394bff0a; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@2c5506e9
2016-05-17 20:45:56,322 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:56,322 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:56,323 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517955973_0001_1_01_000000_0
2016-05-17 20:45:56,323 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517955973_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 20:45:56,324 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517955973_0001_1_01_000000_0
2016-05-17 20:45:56,324 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A3F494A478E84615BDF9C31E63BA6EDC, mem on close (mb), free: 315, total: 653, max: 672
2016-05-17 20:45:56,324 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E97316F327FA4D2D84F09F72A535DE7D
2016-05-17 20:45:56,324 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 20:45:56,326 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 20:45:56,326 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:56,326 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 20:45:56,326 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 20:45:56 Completed running task attempt: attempt_1463517955973_0001_1_01_000000_0
2016-05-17 20:45:56,327 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517955973_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:56,328 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=685244416, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463517956316, Process_Duration=8, Process_End_Time=1463517956324, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=17]]
2016-05-17 20:45:56,329 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:56,329 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:56,330 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:56,329 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A3F494A478E84615BDF9C31E63BA6EDC, taskAttemptId=attempt_1463517955973_0001_1_01_000000_0, startTime=1463517956272, finishTime=1463517956328, timeTaken=56, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=685244416, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517956316, Process_Duration=8, Process_End_Time=1463517956324, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 20:45:56,330 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517955973_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:56,330 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517955973_0001_1_01_000000_0
2016-05-17 20:45:56,330 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517955973_0001_00_000002
2016-05-17 20:45:56,331 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:56,331 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:TASK_FINISHED]: vertexName=A3F494A478E84615BDF9C31E63BA6EDC, taskId=task_1463517955973_0001_1_01_000000, startTime=1463517956272, finishTime=1463517956330, timeTaken=58, status=SUCCEEDED, successfulAttemptID=attempt_1463517955973_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=685244416, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517956316, Process_Duration=8, Process_End_Time=1463517956324, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 20:45:56,331 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:56,331 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517955973_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:56,331 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC] : 1
2016-05-17 20:45:56,331 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC]
2016-05-17 20:45:56,332 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:VERTEX_FINISHED]: vertexName=A3F494A478E84615BDF9C31E63BA6EDC, vertexId=vertex_1463517955973_0001_1_01, initRequestedTime=1463517956148, initedTime=1463517956157, startRequestedTime=1463517956166, startedTime=1463517956166, finishTime=1463517956331, timeTaken=165, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=685244416, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517956316, Process_Duration=8, Process_End_Time=1463517956324, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1463517956272, firstTasksToStart=[ task_1463517955973_0001_1_01_000000 ], lastTaskFinishTime=1463517956328, lastTasksToFinish=[ task_1463517955973_0001_1_01_000000 ], minTaskDuration=56, maxTaskDuration=56, avgTaskDuration=56.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517955973_0001_1_01_000000 ], longestDurationTasks=[ task_1463517955973_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:56,333 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:56,333 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:56,333 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955973_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:56,333 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517955973_0001_00_000002, stoppedTime=1463517956333, exitStatus=0
2016-05-17 20:45:56,333 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955973_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:56,333 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:56,333 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517955973_0001_1
2016-05-17 20:45:56,334 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:56,334 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517955973_0001_1
2016-05-17 20:45:56,334 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: DA0740327CBD4EC1994A080E5DD4B3A3 for vertex: vertex_1463517955973_0001_1_01 [A3F494A478E84615BDF9C31E63BA6EDC]
2016-05-17 20:45:56,341 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517955973_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 20:45:56,341 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517955973_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:56,341 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517955973_0001_00_000002, Exiting
2016-05-17 20:45:56,342 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517955973_0001_00_000002
2016-05-17 20:45:56,342 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517955973_0001_00_000002
2016-05-17 20:45:56,342 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517955973_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:56,342 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517955973_0001_00_000002 completed successfully
2016-05-17 20:45:56,344 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463517955973_0001_1_00 [E97316F327FA4D2D84F09F72A535DE7D]
2016-05-17 20:45:56,345 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517955973_0001_1, queueSize=0
2016-05-17 20:45:56,346 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:56,346 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517955973_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517955973_0001_1, startTime=1463517956146, finishTime=1463517956344, timeTaken=198, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=940, FILE_BYTES_WRITTEN=777, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1370488832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927035912558, Process_Duration=10, Process_End_Time=2927035912568, Read_Duration=2, Tuples_Read=40, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-05-17 20:45:56,346 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517955973_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:56,346 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517955973_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:56,346 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517955973_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:56 Completed Dag: dag_1463517955973_0001_1
2016-05-17 20:45:56,347 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517955973_0001_00_000002
2016-05-17 20:45:56,347 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517955973_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:56,347 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517955973_0001_1_post
2016-05-17 20:45:56,347 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:56,348 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:56,356 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testCount3/count3, applicationId=application_1463517955973_0001
2016-05-17 20:45:56,357 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:56,358 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:56,358 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:56,358 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:56,358 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:56,358 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:56,359 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:56,360 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:56,360 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:56,361 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:56,361 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:56,361 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:56,361 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:56,365 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2393F35DD04D4F36A65748E6C8F8C02A/EA737637910C46578B648B27F774269A/.tez/application_1463517955973_0001
2016-05-17 20:45:56,365 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3/_temporary
2016-05-17 20:45:56,366 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] completed in: 00:00.410
2016-05-17 20:45:56,433 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount3/count3
2016-05-17 20:45:56,456 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:56,456 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:56,490 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 20:45:56,498 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-05-17 20:45:56,498 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:56,498 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:56,498 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:56,512 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-05-17 20:45:56,518 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4
2016-05-17 20:45:56,563 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting
2016-05-17 20:45:56,564 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 20:45:56,564 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4"]
2016-05-17 20:45:56,564 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] parallel execution of steps is enabled: false
2016-05-17 20:45:56,564 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executing total steps: 1
2016-05-17 20:45:56,564 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] allocating management threads: 1
2016-05-17 20:45:56,566 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting step: (1/1) ...latform/testCount4/count4
2016-05-17 20:45:56,566 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:56,578 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tez session mode enabled: true
2016-05-17 20:45:56,582 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:56,582 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:56,582 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:56,593 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789/.tez/application_1463517956582_0001 doesn't exist and is created
2016-05-17 20:45:56,609 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:56,609 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:56,610 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789/.tez/application_1463517956582_0001
2016-05-17 20:45:56,613 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517956582_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:56,623 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:56,624 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:56,627 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:56,627 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:56,627 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789/.tez/application_1463517956582_0001/localmode-log-dir
2016-05-17 20:45:56,628 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789/.tez/application_1463517956582_0001/localmode-log-dir/history.txt.appattempt_1463517956582_0001_000000, maxErrors=10
2016-05-17 20:45:56,628 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:56,628 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517956582_0001_000000, appSubmitTime=1463517956613, launchTime=1463517956613
2016-05-17 20:45:56,637 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:56,638 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:42395
2016-05-17 20:45:56,638 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:56,640 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:56,640 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:56,641 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:56,644 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517956582_0001_000000, startTime=1463517956644
2016-05-17 20:45:56,644 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:56,644 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:56,710 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:56,710 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:56,710 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testCount4/count4, applicationId=application_1463517956582_0001, dagName=[B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789] count/(1/1) ...latform/testCount4/count4
2016-05-17 20:45:56,726 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789] count/(1/1) ...latform/testCount4/count4
2016-05-17 20:45:56,728 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517956582_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789/.tez/application_1463517956582_0001/localmode-log-dir/dag_1463517956582_0001_1.dot
2016-05-17 20:45:56,728 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789/.tez/application_1463517956582_0001/localmode-log-dir/dag_1463517956582_0001_1-tez-dag.pb.txt
2016-05-17 20:45:56,735 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517956582_0001_1
2016-05-17 20:45:56,735 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:56,735 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789] count/(1/1) ...latform/testCount4/count4
2016-05-17 20:45:56 Running Dag: dag_1463517956582_0001_1
2016-05-17 20:45:56,745 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517956582_0001_1, submitTime=1463517956726
2016-05-17 20:45:56,745 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:56,746 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex CEA5F0E3B8F5489594732E8C4C022C3D
2016-05-17 20:45:56,747 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:56,747 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517956582_0001_1, initTime=1463517956746
2016-05-17 20:45:56,747 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:56,747 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517956582_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:56,748 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:DAG_STARTED]: dagID=dag_1463517956582_0001_1, startTime=1463517956748
2016-05-17 20:45:56,748 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:56,748 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:56,748 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517956582_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:56,748 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testCount4/count4, applicationId=application_1463517956582_0001, dagName=[B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789] count/(1/1) ...latform/testCount4/count4
2016-05-17 20:45:56,749 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B3EC9DEBDCA44418BF89E7780E22AEEA : {50961CED08D749CF9E8EE1A94B2CA69A={InputName=50961CED08D749CF9E8EE1A94B2CA69A}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:56,751 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 50961CED08D749CF9E8EE1A94B2CA69A, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:56,751 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA]
2016-05-17 20:45:56,751 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA]
2016-05-17 20:45:56,752 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA]
2016-05-17 20:45:56,752 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA]. Starting root input initializers: 1
2016-05-17 20:45:56,752 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:56,753 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D]
2016-05-17 20:45:56,754 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 50961CED08D749CF9E8EE1A94B2CA69A on vertex vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA]
2016-05-17 20:45:56,755 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 20:45:56,756 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D]
2016-05-17 20:45:56,756 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D]
2016-05-17 20:45:56,756 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D]
2016-05-17 20:45:56,756 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=D4CB83100CAD40BC965E4196415B5BF6, vertexId=vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:56,757 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=D4CB83100CAD40BC965E4196415B5BF6, vertexId=vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D]
2016-05-17 20:45:56,757 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:56,758 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 50961CED08D749CF9E8EE1A94B2CA69A asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:56,758 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:56,759 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:56,759 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=D4CB83100CAD40BC965E4196415B5BF6, vertexName=CEA5F0E3B8F5489594732E8C4C022C3D, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:56,760 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=D4CB83100CAD40BC965E4196415B5BF6, vertexId=vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D]
2016-05-17 20:45:56,761 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:56,761 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-05-17 20:45:56,761 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 50961CED08D749CF9E8EE1A94B2CA69A on vertex vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA]
2016-05-17 20:45:56,762 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CEA5F0E3B8F5489594732E8C4C022C3D, vertexId=vertex_1463517956582_0001_1_01, initRequestedTime=1463517956753, initedTime=1463517956761, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 20:45:56,762 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D] transitioned from NEW to INITED due to event V_INIT
2016-05-17 20:45:56,762 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:56,763 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {50961CED08D749CF9E8EE1A94B2CA69A=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:56,763 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA] parallelism set to 1
2016-05-17 20:45:56,763 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:56,764 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517956582_0001_1_00, eventCount=1
2016-05-17 20:45:56,764 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B3EC9DEBDCA44418BF89E7780E22AEEA, vertexId=vertex_1463517956582_0001_1_00, initRequestedTime=1463517956749, initedTime=1463517956764, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:56,764 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA] with distanceFromRoot: 0
2016-05-17 20:45:56,764 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:56,765 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:56,765 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B3EC9DEBDCA44418BF89E7780E22AEEA
2016-05-17 20:45:56,765 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA]
2016-05-17 20:45:56,766 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:56,766 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517956582_0001_1_00, startRequestedTime=1463517956765, startedTime=1463517956765
2016-05-17 20:45:56,766 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:56,767 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:56,767 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] submitted tez dag to app master: application_1463517956582_0001, with dag id: dag_1463517956582_0001_1
2016-05-17 20:45:56,768 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:TASK_STARTED]: vertexName=B3EC9DEBDCA44418BF89E7780E22AEEA, taskId=task_1463517956582_0001_1_00_000000, scheduledTime=1463517956767, launchTime=1463517956767
2016-05-17 20:45:56,768 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517956582_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:56,769 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463517956582_0001_1_00 for vertex: vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D] numStartedSources: 1 numSources: 1
2016-05-17 20:45:56,769 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D] with distanceFromRoot: 1
2016-05-17 20:45:56,769 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA]
2016-05-17 20:45:56,769 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B3EC9DEBDCA44418BF89E7780E22AEEA in vertex: CEA5F0E3B8F5489594732E8C4C022C3D
2016-05-17 20:45:56,772 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: CEA5F0E3B8F5489594732E8C4C022C3D with 1 source tasks and 1 pending tasks
2016-05-17 20:45:56,773 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517956582_0001_1_01, startRequestedTime=1463517956769, startedTime=1463517956769
2016-05-17 20:45:56,773 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:56,773 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:56,774 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789] count/(1/1) ...latform/testCount4/count4, VertexName: B3EC9DEBDCA44418BF89E7780E22AEEA, VertexParallelism: 1, TaskAttemptID:attempt_1463517956582_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=50961CED08D749CF9E8EE1A94B2CA69A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CEA5F0E3B8F5489594732E8C4C022C3D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:56,774 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517956582_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:56,774 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:56,778 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:56,779 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:56,779 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:56,780 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517956582_0001_00_000001
2016-05-17 20:45:56,781 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517956582_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:56,783 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517956582_0001_00_000001
2016-05-17 20:45:56,783 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:56,784 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517956582_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:56,784 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:56,784 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517956582_0001_00_000001, launchTime=1463517956783
2016-05-17 20:45:56,784 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517956582_0001_00_000001
2016-05-17 20:45:56,785 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517956582_0001_1_00_000000_0] to container: [container_1463517956582_0001_00_000001]
2016-05-17 20:45:56,785 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517956582_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:56,785 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517956582_0001_00_000001 given task: attempt_1463517956582_0001_1_00_000000_0
2016-05-17 20:45:56,785 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517956582_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517956582_0001_1_00_000000_0
2016-05-17 20:45:56,786 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517956582_0001_1_00_000000_0] started. Is using containerId: [container_1463517956582_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:56 Starting to run new task attempt: attempt_1463517956582_0001_1_00_000000_0
2016-05-17 20:45:56,786 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517956582_0001_1_00_000000_0
2016-05-17 20:45:56,787 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:56,787 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:56,787 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:56,787 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:56,787 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:56,787 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:56,787 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789] count/(1/1) ...latform/testCount4/count4, VertexName: B3EC9DEBDCA44418BF89E7780E22AEEA, VertexParallelism: 1, TaskAttemptID:attempt_1463517956582_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=50961CED08D749CF9E8EE1A94B2CA69A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CEA5F0E3B8F5489594732E8C4C022C3D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:56,787 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:56,788 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:56,788 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B3EC9DEBDCA44418BF89E7780E22AEEA, taskAttemptId=attempt_1463517956582_0001_1_00_000000_0, startTime=1463517956786, containerId=container_1463517956582_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517956582_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:56,788 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517956582_0001_1_00_000000_0
2016-05-17 20:45:56,788 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:56,788 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:56,788 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517956582_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:56,789 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CEA5F0E3B8F5489594732E8C4C022C3D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 20:45:56,789 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=50961CED08D749CF9E8EE1A94B2CA69A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:56,789 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:56,793 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:56,792 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:56,792 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517956582_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:56,794 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CEA5F0E3B8F5489594732E8C4C022C3D
2016-05-17 20:45:56,794 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 50961CED08D749CF9E8EE1A94B2CA69A
2016-05-17 20:45:56,794 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 50961CED08D749CF9E8EE1A94B2CA69A
2016-05-17 20:45:56,794 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517956582_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:56,797 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:56,797 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:56,798 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 50961CED08D749CF9E8EE1A94B2CA69A
2016-05-17 20:45:56,798 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 50961CED08D749CF9E8EE1A94B2CA69A
2016-05-17 20:45:56,801 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 20:45:56,801 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: CEA5F0E3B8F5489594732E8C4C022C3D
2016-05-17 20:45:56,801 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CEA5F0E3B8F5489594732E8C4C022C3D
2016-05-17 20:45:56,803 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B3EC9DEBDCA44418BF89E7780E22AEEA, ordinal: 0
2016-05-17 20:45:56,803 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B3EC9DEBDCA44418BF89E7780E22AEEA, mem on start (mb), free: 226, total: 653, max: 672
2016-05-17 20:45:56,803 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:56,803 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:56,803 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:56,803 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:56,803 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:56,803 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:56,803 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:56,804 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 20:45:56,804 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:56,804 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 20:45:56,804 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 50961CED08D749CF9E8EE1A94B2CA69A, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:56,804 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CEA5F0E3B8F5489594732E8C4C022C3D, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 20:45:56,804 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:56,804 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 50961CED08D749CF9E8EE1A94B2CA69A being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:56,805 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:56,805 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:56,805 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 50961CED08D749CF9E8EE1A94B2CA69A
2016-05-17 20:45:56,805 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:56,805 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 50961CED08D749CF9E8EE1A94B2CA69A
2016-05-17 20:45:56,805 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:56,805 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517956582_0001_1_00_000000_0
2016-05-17 20:45:56,805 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:56,808 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3bfd4ff0
2016-05-17 20:45:56,808 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 20:45:56,809 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: 50961CED08D749CF9E8EE1A94B2CA69A
2016-05-17 20:45:56,809 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: B00C424EED6E435BA56DD6DF2A1DB36B
2016-05-17 20:45:56,809 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] B00C424EED6E435BA56DD6DF2A1DB36B
2016-05-17 20:45:56,810 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3bfd4ff0
2016-05-17 20:45:56,810 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:56,810 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:56,811 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@52950d16; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@378e49b5; comparator=cascading.tuple.hadoop.util.TupleComparator@7e432a13; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 20:45:56,812 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 20:45:56,813 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 20:45:56,813 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:56,835 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 20:45:56,835 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 20:45:56,835 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 20:45:56,835 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 20:45:56,835 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 20:45:56,836 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-05-17 20:45:56,836 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:56,837 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B3EC9DEBDCA44418BF89E7780E22AEEA, all 1 inputs ready in: 00:00:00.001
2016-05-17 20:45:56,843 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517956582_0001_1_00_000000_0
2016-05-17 20:45:56,843 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B3EC9DEBDCA44418BF89E7780E22AEEA, mem on close (mb), free: 322, total: 654, max: 672
2016-05-17 20:45:56,843 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 20:45:56,843 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 20:45:56,844 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2016-05-17 20:45:56,844 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 20:45:56,848 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 20:45:56 Completed running task attempt: attempt_1463517956582_0001_1_00_000000_0
2016-05-17 20:45:56,851 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517956582_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:56,852 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517956582_0001_1_00, eventCount=1
2016-05-17 20:45:56,852 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686292992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463517956837, Process_Duration=6, Process_End_Time=1463517956843, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 20:45:56,852 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:56,852 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:56,852 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B3EC9DEBDCA44418BF89E7780E22AEEA, taskAttemptId=attempt_1463517956582_0001_1_00_000000_0, startTime=1463517956786, finishTime=1463517956852, timeTaken=66, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686292992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517956837, Process_Duration=6, Process_End_Time=1463517956843, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 20:45:56,853 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:56,853 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517956582_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:56,853 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517956582_0001_1_00_000000_0
2016-05-17 20:45:56,853 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517956582_0001_00_000001
2016-05-17 20:45:56,854 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:56,854 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:56,854 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:TASK_FINISHED]: vertexName=B3EC9DEBDCA44418BF89E7780E22AEEA, taskId=task_1463517956582_0001_1_00_000000, startTime=1463517956786, finishTime=1463517956853, timeTaken=67, status=SUCCEEDED, successfulAttemptID=attempt_1463517956582_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686292992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517956837, Process_Duration=6, Process_End_Time=1463517956843, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 20:45:56,855 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517956582_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:56,855 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA] : 1
2016-05-17 20:45:56,855 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA]
2016-05-17 20:45:56,856 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:VERTEX_FINISHED]: vertexName=B3EC9DEBDCA44418BF89E7780E22AEEA, vertexId=vertex_1463517956582_0001_1_00, initRequestedTime=1463517956749, initedTime=1463517956764, startRequestedTime=1463517956765, startedTime=1463517956765, finishTime=1463517956855, timeTaken=90, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686292992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517956837, Process_Duration=6, Process_End_Time=1463517956843, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463517956786, firstTasksToStart=[ task_1463517956582_0001_1_00_000000 ], lastTaskFinishTime=1463517956852, lastTasksToFinish=[ task_1463517956582_0001_1_00_000000 ], minTaskDuration=66, maxTaskDuration=66, avgTaskDuration=66.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517956582_0001_1_00_000000 ], longestDurationTasks=[ task_1463517956582_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:56,857 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:56,857 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517956582_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:56,857 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517956582_0001_00_000001, stoppedTime=1463517956857, exitStatus=0
2016-05-17 20:45:56,857 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517956582_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:56,857 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D] attempt: attempt_1463517956582_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 20:45:56,857 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: CEA5F0E3B8F5489594732E8C4C022C3D
2016-05-17 20:45:56,857 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D]
2016-05-17 20:45:56,858 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:56,858 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:TASK_STARTED]: vertexName=CEA5F0E3B8F5489594732E8C4C022C3D, taskId=task_1463517956582_0001_1_01_000000, scheduledTime=1463517956858, launchTime=1463517956858
2016-05-17 20:45:56,858 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517956582_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:56,858 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463517956582_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 20:45:56,858 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:56,858 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D]
2016-05-17 20:45:56,859 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:56,859 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:56,859 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517956582_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 20:45:56,860 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789] count/(1/1) ...latform/testCount4/count4, VertexName: CEA5F0E3B8F5489594732E8C4C022C3D, VertexParallelism: 1, TaskAttemptID:attempt_1463517956582_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B3EC9DEBDCA44418BF89E7780E22AEEA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=D4CB83100CAD40BC965E4196415B5BF6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:56,860 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517956582_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:56,860 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:56,860 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517956582_0001_00_000002
2016-05-17 20:45:56,860 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517956582_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:56,862 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517956582_0001_00_000002
2016-05-17 20:45:56,862 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:56,862 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517956582_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:56,862 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:56,862 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517956582_0001_00_000002, launchTime=1463517956862
2016-05-17 20:45:56,863 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517956582_0001_00_000002
2016-05-17 20:45:56,863 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517956582_0001_1_01_000000_0] to container: [container_1463517956582_0001_00_000002]
2016-05-17 20:45:56,863 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517956582_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:56,864 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517956582_0001_00_000002 given task: attempt_1463517956582_0001_1_01_000000_0
2016-05-17 20:45:56,864 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517956582_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517956582_0001_1_01_000000_0
2016-05-17 20:45:56,864 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517956582_0001_1_01_000000_0] started. Is using containerId: [container_1463517956582_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 20:45:56,864 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:56,864 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517956582_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:56,864 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CEA5F0E3B8F5489594732E8C4C022C3D, taskAttemptId=attempt_1463517956582_0001_1_01_000000_0, startTime=1463517956864, containerId=container_1463517956582_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517956582_0001_00_000002/teamcity, completedLogs=
2016-05-17 20:45:56,865 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517956582_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:56,865 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517956582_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:56,865 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517956582_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:56,865 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517956582_0001_00_000001, Exiting
2016-05-17 20:45:56,865 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517956582_0001_00_000001
2016-05-17 20:45:56 Starting to run new task attempt: attempt_1463517956582_0001_1_01_000000_0
2016-05-17 20:45:56,865 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517956582_0001_00_000001
2016-05-17 20:45:56,865 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517956582_0001_1_01_000000_0
2016-05-17 20:45:56,865 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:56,865 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:56,865 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:56,866 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:56,866 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:56,866 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:56,866 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789] count/(1/1) ...latform/testCount4/count4, VertexName: CEA5F0E3B8F5489594732E8C4C022C3D, VertexParallelism: 1, TaskAttemptID:attempt_1463517956582_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B3EC9DEBDCA44418BF89E7780E22AEEA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=D4CB83100CAD40BC965E4196415B5BF6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:56,866 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:56,867 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517956582_0001_1_01_000000_0
2016-05-17 20:45:56,867 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:56,867 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:56,868 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517956582_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 20:45:56,867 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517956582_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:56,868 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517956582_0001_00_000001 completed successfully
2016-05-17 20:45:56,868 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B3EC9DEBDCA44418BF89E7780E22AEEA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 20:45:56,869 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:56,869 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B3EC9DEBDCA44418BF89E7780E22AEEA
2016-05-17 20:45:56,870 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 20:45:56,871 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517956582_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 20:45:56,871 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 20:45:56,871 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B3EC9DEBDCA44418BF89E7780E22AEEA
2016-05-17 20:45:56,872 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B3EC9DEBDCA44418BF89E7780E22AEEA
2016-05-17 20:45:56,872 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:56,872 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D4CB83100CAD40BC965E4196415B5BF6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:56,873 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:56,873 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D4CB83100CAD40BC965E4196415B5BF6
2016-05-17 20:45:56,873 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:56,873 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: D4CB83100CAD40BC965E4196415B5BF6
2016-05-17 20:45:56,875 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:56,881 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:56,882 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D4CB83100CAD40BC965E4196415B5BF6
2016-05-17 20:45:56,883 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CEA5F0E3B8F5489594732E8C4C022C3D, ordinal: 1
2016-05-17 20:45:56,883 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CEA5F0E3B8F5489594732E8C4C022C3D, mem on start (mb), free: 316, total: 654, max: 672
2016-05-17 20:45:56,883 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:56,884 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:56,884 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:56,884 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:56,885 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:56,885 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 20:45:56,885 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:56,886 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:56,886 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:56,887 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 20:45:56,887 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B3EC9DEBDCA44418BF89E7780E22AEEA, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 20:45:56,887 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D4CB83100CAD40BC965E4196415B5BF6, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:56,888 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:56,888 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B3EC9DEBDCA44418BF89E7780E22AEEA being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:56,888 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B3EC9DEBDCA44418BF89E7780E22AEEA
2016-05-17 20:45:56,889 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:56,889 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:56,889 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:56,890 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 20:45:56,890 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B3EC9DEBDCA44418BF89E7780E22AEEA with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 20:45:56,890 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 20:45:56,891 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 20:45:56,891 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 20:45:56,892 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B3EC9DEBDCA44418BF89E7780E22AEEA
2016-05-17 20:45:56,893 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:56,893 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517956582_0001_1_01_000000_0
2016-05-17 20:45:56,894 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463517956582_0001_1_00_000000_0_10079, runDuration: 53117, ]
2016-05-17 20:45:56,896 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-05-17 20:45:56,896 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463517956582_0001_1_00_000000_0_10079] to DISK_DIRECT, CompressedSize=463, DecompressedSize=459,EndTime=1463517956896, TimeTaken=1, Rate=0.44 MB/s
2016-05-17 20:45:56,897 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B3EC9DEBDCA44418BF89E7780E22AEEA
2016-05-17 20:45:56,898 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 20:45:56,898 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [B3EC9DEBDCA44418BF89E7780E22AEEA] #13 in 3ms
2016-05-17 20:45:56,899 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: B00C424EED6E435BA56DD6DF2A1DB36B
2016-05-17 20:45:56,899 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4"], id: D4CB83100CAD40BC965E4196415B5BF6
2016-05-17 20:45:56,900 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4"]
2016-05-17 20:45:56,900 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] B00C424EED6E435BA56DD6DF2A1DB36B, for 1 inputs
2016-05-17 20:45:56,899 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B3EC9DEBDCA44418BF89E7780E22AEEA] #13, status:false, isInterrupted:false
2016-05-17 20:45:56,901 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 20:45:56,902 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 463 bytes from disk
2016-05-17 20:45:56,903 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:56,903 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 20:45:56,903 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-05-17 20:45:56,904 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CEA5F0E3B8F5489594732E8C4C022C3D, all 1 inputs ready in: 00:00:00.004
2016-05-17 20:45:56,904 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B3EC9DEBDCA44418BF89E7780E22AEEA
2016-05-17 20:45:56,906 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@1f3ad2aa; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@4edca6ea
2016-05-17 20:45:56,906 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 20:45:56,909 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:56,910 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:56,910 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517956582_0001_1_01_000000_0
2016-05-17 20:45:56,910 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517956582_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 20:45:56,911 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517956582_0001_1_01_000000_0
2016-05-17 20:45:56,911 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CEA5F0E3B8F5489594732E8C4C022C3D, mem on close (mb), free: 313, total: 654, max: 672
2016-05-17 20:45:56,911 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B3EC9DEBDCA44418BF89E7780E22AEEA
2016-05-17 20:45:56,912 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 20:45:56,913 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 20:45:56,913 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:56,913 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 20:45:56,914 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 20:45:56 Completed running task attempt: attempt_1463517956582_0001_1_01_000000_0
2016-05-17 20:45:56,915 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517956582_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:56,915 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686292992, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463517956904, Process_Duration=7, Process_End_Time=1463517956911, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=17]]
2016-05-17 20:45:56,916 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:56,916 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CEA5F0E3B8F5489594732E8C4C022C3D, taskAttemptId=attempt_1463517956582_0001_1_01_000000_0, startTime=1463517956864, finishTime=1463517956915, timeTaken=51, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686292992, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517956904, Process_Duration=7, Process_End_Time=1463517956911, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 20:45:56,916 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517956582_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:56,916 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517956582_0001_1_01_000000_0
2016-05-17 20:45:56,916 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:56,917 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:56,917 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:56,917 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:56,917 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:TASK_FINISHED]: vertexName=CEA5F0E3B8F5489594732E8C4C022C3D, taskId=task_1463517956582_0001_1_01_000000, startTime=1463517956864, finishTime=1463517956916, timeTaken=52, status=SUCCEEDED, successfulAttemptID=attempt_1463517956582_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686292992, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517956904, Process_Duration=7, Process_End_Time=1463517956911, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-17 20:45:56,917 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517956582_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:56,917 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D] : 1
2016-05-17 20:45:56,917 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D]
2016-05-17 20:45:56,919 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:56,917 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517956582_0001_00_000002
2016-05-17 20:45:56,920 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:VERTEX_FINISHED]: vertexName=CEA5F0E3B8F5489594732E8C4C022C3D, vertexId=vertex_1463517956582_0001_1_01, initRequestedTime=1463517956753, initedTime=1463517956761, startRequestedTime=1463517956769, startedTime=1463517956769, finishTime=1463517956918, timeTaken=149, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686292992, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517956904, Process_Duration=7, Process_End_Time=1463517956911, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1463517956864, firstTasksToStart=[ task_1463517956582_0001_1_01_000000 ], lastTaskFinishTime=1463517956915, lastTasksToFinish=[ task_1463517956582_0001_1_01_000000 ], minTaskDuration=51, maxTaskDuration=51, avgTaskDuration=51.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517956582_0001_1_01_000000 ], longestDurationTasks=[ task_1463517956582_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:56,920 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:56,920 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517956582_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:56,920 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517956582_0001_00_000002, stoppedTime=1463517956920, exitStatus=0
2016-05-17 20:45:56,921 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517956582_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:56,921 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:56,921 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517956582_0001_1
2016-05-17 20:45:56,921 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517956582_0001_1
2016-05-17 20:45:56,921 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463517956582_0001_1_00 [B3EC9DEBDCA44418BF89E7780E22AEEA]
2016-05-17 20:45:56,921 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: D4CB83100CAD40BC965E4196415B5BF6 for vertex: vertex_1463517956582_0001_1_01 [CEA5F0E3B8F5489594732E8C4C022C3D]
2016-05-17 20:45:56,922 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:56,929 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517956582_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 20:45:56,929 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517956582_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:56,929 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517956582_0001_00_000002, Exiting
2016-05-17 20:45:56,929 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517956582_0001_00_000002
2016-05-17 20:45:56,929 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517956582_0001_00_000002
2016-05-17 20:45:56,930 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517956582_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:56,930 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517956582_0001_00_000002 completed successfully
2016-05-17 20:45:56,931 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517956582_0001_1, queueSize=0
2016-05-17 20:45:56,931 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:56,932 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517956582_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517956582_0001_1, startTime=1463517956748, finishTime=1463517956928, timeTaken=180, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=940, FILE_BYTES_WRITTEN=1297, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1372585984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927035913741, Process_Duration=13, Process_End_Time=2927035913754, Read_Duration=6, Tuples_Read=40, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-05-17 20:45:56,932 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517956582_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:56,933 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517956582_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:56,933 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517956582_0001_00_000002
2016-05-17 20:45:56,935 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517956582_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:56 Completed Dag: dag_1463517956582_0001_1
2016-05-17 20:45:56,937 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517956582_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:56,940 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517956582_0001_1_post
2016-05-17 20:45:56,940 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:56,940 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:56,940 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testCount4/count4, applicationId=application_1463517956582_0001
2016-05-17 20:45:56,942 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:56,942 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:56,942 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:56,942 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:56,943 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:56,944 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:56,944 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:56,944 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:56,944 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:56,944 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:56,945 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:56,945 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:56,948 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:56,950 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B642C850CB0A4FD7A5678F91F501B25A/DE11FCD80C464F918C8AB25271804789/.tez/application_1463517956582_0001
2016-05-17 20:45:56,950 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4/_temporary
2016-05-17 20:45:56,951 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] completed in: 00:00.387
2016-05-17 20:45:57,019 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount4/count4
2016-05-17 20:45:57,041 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:57,041 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:57,071 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-05-17 20:45:57,078 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-17 20:45:57,079 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 20:45:57,079 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 20:45:57,080 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:57,094 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 20:45:57,094 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter
2016-05-17 20:45:57,103 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 20:45:57,103 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:57,103 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter"]
2016-05-17 20:45:57,104 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 20:45:57,104 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 20:45:57,104 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 20:45:57,105 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...latform/testFilter/filter
2016-05-17 20:45:57,105 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:57,117 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 20:45:57,121 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:57,122 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:57,122 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:57,131 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/CFA350850F00441585FE202422EC13D5/E26143F9E117402A8605756E14235C83/.tez/application_1463517957121_0001 doesn't exist and is created
2016-05-17 20:45:57,146 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:57,146 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:57,146 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/CFA350850F00441585FE202422EC13D5/E26143F9E117402A8605756E14235C83/.tez/application_1463517957121_0001
2016-05-17 20:45:57,149 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517957121_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:57,159 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:57,160 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:57,161 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:57,161 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:57,161 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/CFA350850F00441585FE202422EC13D5/E26143F9E117402A8605756E14235C83/.tez/application_1463517957121_0001/localmode-log-dir
2016-05-17 20:45:57,162 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/CFA350850F00441585FE202422EC13D5/E26143F9E117402A8605756E14235C83/.tez/application_1463517957121_0001/localmode-log-dir/history.txt.appattempt_1463517957121_0001_000000, maxErrors=10
2016-05-17 20:45:57,162 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:57,162 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517957121_0001_000000, appSubmitTime=1463517957149, launchTime=1463517957149
2016-05-17 20:45:57,165 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:57,165 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:57,166 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:57,167 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:57,170 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:51116
2016-05-17 20:45:57,171 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:57,173 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517957121_0001_000000, startTime=1463517957173
2016-05-17 20:45:57,173 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:57,173 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:57,246 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:57,246 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:57,246 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testFilter/filter, applicationId=application_1463517957121_0001, dagName=[CFA350850F00441585FE202422EC13D5/E26143F9E117402A8605756E14235C83] test/(1/1) ...latform/testFilter/filter
2016-05-17 20:45:57,261 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [CFA350850F00441585FE202422EC13D5/E26143F9E117402A8605756E14235C83] test/(1/1) ...latform/testFilter/filter
2016-05-17 20:45:57,263 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517957121_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/CFA350850F00441585FE202422EC13D5/E26143F9E117402A8605756E14235C83/.tez/application_1463517957121_0001/localmode-log-dir/dag_1463517957121_0001_1.dot
2016-05-17 20:45:57,263 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/CFA350850F00441585FE202422EC13D5/E26143F9E117402A8605756E14235C83/.tez/application_1463517957121_0001/localmode-log-dir/dag_1463517957121_0001_1-tez-dag.pb.txt
2016-05-17 20:45:57,268 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517957121_0001_1
2016-05-17 20:45:57,268 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:57,268 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [CFA350850F00441585FE202422EC13D5/E26143F9E117402A8605756E14235C83] test/(1/1) ...latform/testFilter/filter
2016-05-17 20:45:57 Running Dag: dag_1463517957121_0001_1
2016-05-17 20:45:57,278 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:57,279 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517957121_0001_1, submitTime=1463517957262
2016-05-17 20:45:57,279 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E5134DB3CCA440B1BC64C1CBE0ECF57E
2016-05-17 20:45:57,279 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:57,280 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:57,280 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517957121_0001_1, initTime=1463517957279
2016-05-17 20:45:57,280 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517957121_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:57,281 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:DAG_STARTED]: dagID=dag_1463517957121_0001_1, startTime=1463517957281
2016-05-17 20:45:57,281 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:57,281 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:57,281 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517957121_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:57,282 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E5134DB3CCA440B1BC64C1CBE0ECF57E : {96B49CDB7AA747DF833FC44A935DBF0D={InputName=96B49CDB7AA747DF833FC44A935DBF0D}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:57,282 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 96B49CDB7AA747DF833FC44A935DBF0D, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:57,282 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testFilter/filter, applicationId=application_1463517957121_0001, dagName=[CFA350850F00441585FE202422EC13D5/E26143F9E117402A8605756E14235C83] test/(1/1) ...latform/testFilter/filter
2016-05-17 20:45:57,282 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E]
2016-05-17 20:45:57,282 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E]
2016-05-17 20:45:57,282 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E]
2016-05-17 20:45:57,282 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E]. Starting root input initializers: 1
2016-05-17 20:45:57,283 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:57,284 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 96B49CDB7AA747DF833FC44A935DBF0D on vertex vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E]
2016-05-17 20:45:57,286 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:57,286 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 96B49CDB7AA747DF833FC44A935DBF0D asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:57,287 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:57,287 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:57,288 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 20:45:57,289 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 96B49CDB7AA747DF833FC44A935DBF0D on vertex vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E]
2016-05-17 20:45:57,289 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:57,289 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {96B49CDB7AA747DF833FC44A935DBF0D=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:57,289 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E] parallelism set to 1
2016-05-17 20:45:57,289 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:57,290 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517957121_0001_1_00, eventCount=1
2016-05-17 20:45:57,290 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E]
2016-05-17 20:45:57,290 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=FA491E0CA247478F8B6C06CEAB003090, vertexId=vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:57,290 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=FA491E0CA247478F8B6C06CEAB003090, vertexId=vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E]
2016-05-17 20:45:57,292 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=FA491E0CA247478F8B6C06CEAB003090, vertexName=E5134DB3CCA440B1BC64C1CBE0ECF57E, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:57,292 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=FA491E0CA247478F8B6C06CEAB003090, vertexId=vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E]
2016-05-17 20:45:57,293 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E5134DB3CCA440B1BC64C1CBE0ECF57E, vertexId=vertex_1463517957121_0001_1_00, initRequestedTime=1463517957282, initedTime=1463517957293, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:57,293 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:57,294 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E] with distanceFromRoot: 0
2016-05-17 20:45:57,294 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:57,294 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E5134DB3CCA440B1BC64C1CBE0ECF57E
2016-05-17 20:45:57,294 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E]
2016-05-17 20:45:57,295 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:57,295 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517957121_0001_1_00, startRequestedTime=1463517957294, startedTime=1463517957294
2016-05-17 20:45:57,295 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:57,296 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:57,296 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:TASK_STARTED]: vertexName=E5134DB3CCA440B1BC64C1CBE0ECF57E, taskId=task_1463517957121_0001_1_00_000000, scheduledTime=1463517957296, launchTime=1463517957296
2016-05-17 20:45:57,296 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517957121_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:57,297 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E]
2016-05-17 20:45:57,297 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [CFA350850F00441585FE202422EC13D5/E26143F9E117402A8605756E14235C83] test/(1/1) ...latform/testFilter/filter, VertexName: E5134DB3CCA440B1BC64C1CBE0ECF57E, VertexParallelism: 1, TaskAttemptID:attempt_1463517957121_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=96B49CDB7AA747DF833FC44A935DBF0D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FA491E0CA247478F8B6C06CEAB003090, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:57,297 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517957121_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:57,298 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:57,299 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:57,299 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:57,299 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:57,300 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517957121_0001_00_000001
2016-05-17 20:45:57,300 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957121_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:57,302 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517957121_0001_00_000001
2016-05-17 20:45:57,302 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:57,303 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957121_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:57,303 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517957121_0001_00_000001, launchTime=1463517957303
2016-05-17 20:45:57,303 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:57,304 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517957121_0001_00_000001
2016-05-17 20:45:57,304 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517957121_0001_1_00_000000_0] to container: [container_1463517957121_0001_00_000001]
2016-05-17 20:45:57,304 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957121_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:57,305 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463517957121_0001, with dag id: dag_1463517957121_0001_1
2016-05-17 20:45:57,305 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517957121_0001_00_000001 given task: attempt_1463517957121_0001_1_00_000000_0
2016-05-17 20:45:57,306 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517957121_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517957121_0001_1_00_000000_0
2016-05-17 20:45:57,307 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517957121_0001_1_00_000000_0] started. Is using containerId: [container_1463517957121_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:57 Starting to run new task attempt: attempt_1463517957121_0001_1_00_000000_0
2016-05-17 20:45:57,307 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517957121_0001_1_00_000000_0
2016-05-17 20:45:57,307 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:57,307 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:57,307 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E5134DB3CCA440B1BC64C1CBE0ECF57E, taskAttemptId=attempt_1463517957121_0001_1_00_000000_0, startTime=1463517957306, containerId=container_1463517957121_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517957121_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:57,308 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517957121_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:57,307 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:57,308 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:57,308 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:57,309 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:57,309 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:57,309 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [CFA350850F00441585FE202422EC13D5/E26143F9E117402A8605756E14235C83] test/(1/1) ...latform/testFilter/filter, VertexName: E5134DB3CCA440B1BC64C1CBE0ECF57E, VertexParallelism: 1, TaskAttemptID:attempt_1463517957121_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=96B49CDB7AA747DF833FC44A935DBF0D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FA491E0CA247478F8B6C06CEAB003090, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:57,309 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:57,309 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517957121_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:57,310 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517957121_0001_1_00_000000_0
2016-05-17 20:45:57,310 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:57,310 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:57,311 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517957121_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:57,311 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:57,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=96B49CDB7AA747DF833FC44A935DBF0D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:57,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:57,313 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 96B49CDB7AA747DF833FC44A935DBF0D
2016-05-17 20:45:57,313 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 96B49CDB7AA747DF833FC44A935DBF0D
2016-05-17 20:45:57,314 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:57,316 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:57,317 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 96B49CDB7AA747DF833FC44A935DBF0D
2016-05-17 20:45:57,317 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 96B49CDB7AA747DF833FC44A935DBF0D
2016-05-17 20:45:57,317 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FA491E0CA247478F8B6C06CEAB003090, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:57,318 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:57,318 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FA491E0CA247478F8B6C06CEAB003090
2016-05-17 20:45:57,318 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:57,318 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: FA491E0CA247478F8B6C06CEAB003090
2016-05-17 20:45:57,320 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E5134DB3CCA440B1BC64C1CBE0ECF57E, ordinal: 0
2016-05-17 20:45:57,320 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E5134DB3CCA440B1BC64C1CBE0ECF57E, mem on start (mb), free: 241, total: 654, max: 672
2016-05-17 20:45:57,320 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:57,321 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:57,321 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:57,330 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:57,330 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FA491E0CA247478F8B6C06CEAB003090
2016-05-17 20:45:57,330 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:57,330 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:57,330 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:57,330 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:57,331 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:57,331 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:57,331 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:57,331 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 20:45:57,331 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 96B49CDB7AA747DF833FC44A935DBF0D, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:57,331 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FA491E0CA247478F8B6C06CEAB003090, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:57,331 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:57,331 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 96B49CDB7AA747DF833FC44A935DBF0D being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:57,332 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:57,332 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:57,332 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 96B49CDB7AA747DF833FC44A935DBF0D
2016-05-17 20:45:57,332 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:57,332 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 96B49CDB7AA747DF833FC44A935DBF0D
2016-05-17 20:45:57,333 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:57,333 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517957121_0001_1_00_000000_0
2016-05-17 20:45:57,336 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:57,337 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 96B49CDB7AA747DF833FC44A935DBF0D
2016-05-17 20:45:57,337 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter"], id: FA491E0CA247478F8B6C06CEAB003090
2016-05-17 20:45:57,337 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter"]
2016-05-17 20:45:57,337 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:57,337 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:57,338 INFO input.MRInput (MRInput.java:checkAndAwaitRecordReaderInitialization(597)) - Awaiting RecordReader initialization
2016-05-17 20:45:57,338 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:57,338 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2e5524e
2016-05-17 20:45:57,339 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2e5524e
2016-05-17 20:45:57,340 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:57,340 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:57,340 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E5134DB3CCA440B1BC64C1CBE0ECF57E, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:57,342 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:57,342 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:57,343 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517957121_0001_1_00_000000_0
2016-05-17 20:45:57,343 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517957121_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 20:45:57,344 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517957121_0001_1_00_000000_0
2016-05-17 20:45:57,344 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E5134DB3CCA440B1BC64C1CBE0ECF57E, mem on close (mb), free: 240, total: 654, max: 672
2016-05-17 20:45:57 Completed running task attempt: attempt_1463517957121_0001_1_00_000000_0
2016-05-17 20:45:57,345 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517957121_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:57,346 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=1027, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686292992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3][cascading.flow.SliceCounters Process_Begin_Time=1463517957340, Process_Duration=4, Process_End_Time=1463517957344, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10, Tuples_Written=3]]
2016-05-17 20:45:57,346 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E5134DB3CCA440B1BC64C1CBE0ECF57E, taskAttemptId=attempt_1463517957121_0001_1_00_000000_0, startTime=1463517957306, finishTime=1463517957346, timeTaken=40, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=1027, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686292992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1463517957340, Process_Duration=4, Process_End_Time=1463517957344, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3
2016-05-17 20:45:57,346 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:57,347 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517957121_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:57,347 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:57,347 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517957121_0001_1_00_000000_0
2016-05-17 20:45:57,347 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:57,347 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:57,348 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517957121_0001_00_000001
2016-05-17 20:45:57,348 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:TASK_FINISHED]: vertexName=E5134DB3CCA440B1BC64C1CBE0ECF57E, taskId=task_1463517957121_0001_1_00_000000, startTime=1463517957306, finishTime=1463517957347, timeTaken=41, status=SUCCEEDED, successfulAttemptID=attempt_1463517957121_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=1027, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686292992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1463517957340, Process_Duration=4, Process_End_Time=1463517957344, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3
2016-05-17 20:45:57,348 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:57,348 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517957121_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:57,349 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E] : 1
2016-05-17 20:45:57,349 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E]
2016-05-17 20:45:57,350 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:57,350 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:VERTEX_FINISHED]: vertexName=E5134DB3CCA440B1BC64C1CBE0ECF57E, vertexId=vertex_1463517957121_0001_1_00, initRequestedTime=1463517957282, initedTime=1463517957293, startRequestedTime=1463517957294, startedTime=1463517957294, finishTime=1463517957349, timeTaken=55, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=1027, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686292992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1463517957340, Process_Duration=4, Process_End_Time=1463517957344, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3, vertexStats=firstTaskStartTime=1463517957306, firstTasksToStart=[ task_1463517957121_0001_1_00_000000 ], lastTaskFinishTime=1463517957346, lastTasksToFinish=[ task_1463517957121_0001_1_00_000000 ], minTaskDuration=40, maxTaskDuration=40, avgTaskDuration=40.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517957121_0001_1_00_000000 ], longestDurationTasks=[ task_1463517957121_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:57,350 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:57,350 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957121_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:57,350 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517957121_0001_00_000001, stoppedTime=1463517957350, exitStatus=0
2016-05-17 20:45:57,350 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:57,351 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957121_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:57,351 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 20:45:57,351 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517957121_0001_1
2016-05-17 20:45:57,351 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517957121_0001_1
2016-05-17 20:45:57,351 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: FA491E0CA247478F8B6C06CEAB003090 for vertex: vertex_1463517957121_0001_1_00 [E5134DB3CCA440B1BC64C1CBE0ECF57E]
2016-05-17 20:45:57,358 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517957121_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 20:45:57,359 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517957121_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:57,359 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517957121_0001_00_000001, Exiting
2016-05-17 20:45:57,359 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517957121_0001_00_000001
2016-05-17 20:45:57,359 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517957121_0001_00_000001
2016-05-17 20:45:57,360 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517957121_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:57,360 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517957121_0001_00_000001 completed successfully
2016-05-17 20:45:57,361 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517957121_0001_1, queueSize=0
2016-05-17 20:45:57,361 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:57,362 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957121_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517957121_0001_1, startTime=1463517957281, finishTime=1463517957358, timeTaken=77, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=1027, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686292992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1463517957340, Process_Duration=4, Process_End_Time=1463517957344, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3
2016-05-17 20:45:57,362 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517957121_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:57,362 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517957121_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:57,363 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517957121_0001_00_000001
2016-05-17 20:45:57,363 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957121_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:57 Completed Dag: dag_1463517957121_0001_1
2016-05-17 20:45:57,364 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517957121_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:57,365 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517957121_0001_1_post
2016-05-17 20:45:57,365 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:57,365 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:57,368 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testFilter/filter, applicationId=application_1463517957121_0001
2016-05-17 20:45:57,370 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:57,370 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:57,370 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:57,370 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:57,370 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:57,371 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:57,371 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:57,371 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:57,372 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:57,372 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:57,372 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:57,373 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:57,373 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:57,376 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/CFA350850F00441585FE202422EC13D5/E26143F9E117402A8605756E14235C83/.tez/application_1463517957121_0001
2016-05-17 20:45:57,377 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter/_temporary
2016-05-17 20:45:57,377 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.274
2016-05-17 20:45:57,410 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testFilter/filter
2016-05-17 20:45:57,435 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 20:45:57,435 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 20:45:57,458 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 20:45:57,458 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 20:45:57,471 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-17 20:45:57,486 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.028
2016-05-17 20:45:57,487 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:57,487 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 20:45:57,488 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 20:45:57,503 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 20:45:57,518 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple
2016-05-17 20:45:57,563 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 20:45:57,564 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:57,565 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple"]
2016-05-17 20:45:57,565 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 20:45:57,565 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 20:45:57,566 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 20:45:57,567 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...latform/testSimple/simple
2016-05-17 20:45:57,567 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 20:45:57,578 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 20:45:57,582 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:57,583 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 20:45:57,583 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 20:45:57,592 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E/.tez/application_1463517957583_0001 doesn't exist and is created
2016-05-17 20:45:57,607 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 20:45:57,607 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 20:45:57,608 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E/.tez/application_1463517957583_0001
2016-05-17 20:45:57,610 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463517957583_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 20:45:57,619 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 20:45:57,620 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 20:45:57,621 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 20:45:57,622 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 20:45:57,622 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E/.tez/application_1463517957583_0001/localmode-log-dir
2016-05-17 20:45:57,622 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E/.tez/application_1463517957583_0001/localmode-log-dir/history.txt.appattempt_1463517957583_0001_000000, maxErrors=10
2016-05-17 20:45:57,622 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 20:45:57,623 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463517957583_0001_000000, appSubmitTime=1463517957609, launchTime=1463517957610
2016-05-17 20:45:57,625 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 20:45:57,628 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 20:45:57,630 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 20:45:57,630 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 20:45:57,631 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:42671
2016-05-17 20:45:57,632 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 20:45:57,636 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463517957583_0001_000000, startTime=1463517957636
2016-05-17 20:45:57,636 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 20:45:57,636 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 20:45:57,707 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 20:45:57,707 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 20:45:57,707 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testSimple/simple, applicationId=application_1463517957583_0001, dagName=[D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E] test/(1/1) ...latform/testSimple/simple
2016-05-17 20:45:57,723 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E] test/(1/1) ...latform/testSimple/simple
2016-05-17 20:45:57,725 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463517957583_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E/.tez/application_1463517957583_0001/localmode-log-dir/dag_1463517957583_0001_1.dot
2016-05-17 20:45:57,725 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E/.tez/application_1463517957583_0001/localmode-log-dir/dag_1463517957583_0001_1-tez-dag.pb.txt
2016-05-17 20:45:57,736 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517957583_0001_1
2016-05-17 20:45:57,736 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:57,737 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E] test/(1/1) ...latform/testSimple/simple
2016-05-17 20:45:57 Running Dag: dag_1463517957583_0001_1
2016-05-17 20:45:57,749 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 20:45:57,749 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463517957583_0001_1, submitTime=1463517957723
2016-05-17 20:45:57,750 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D32E1B14CEFD419D9F85225A98723848
2016-05-17 20:45:57,750 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 20:45:57,751 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 20:45:57,752 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463517957583_0001_1, initTime=1463517957750
2016-05-17 20:45:57,752 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517957583_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 20:45:57,752 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:DAG_STARTED]: dagID=dag_1463517957583_0001_1, startTime=1463517957752
2016-05-17 20:45:57,752 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 20:45:57,752 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 20:45:57,753 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517957583_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 20:45:57,753 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CE088C9F6B3E48728F84F5607D3748C4 : {E9E5621A47A145F1B219FD77E7B7324E={InputName=E9E5621A47A145F1B219FD77E7B7324E}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 20:45:57,753 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: E9E5621A47A145F1B219FD77E7B7324E, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 20:45:57,753 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testSimple/simple, applicationId=application_1463517957583_0001, dagName=[D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E] test/(1/1) ...latform/testSimple/simple
2016-05-17 20:45:57,753 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4]
2016-05-17 20:45:57,754 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4]
2016-05-17 20:45:57,754 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4]
2016-05-17 20:45:57,754 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4]. Starting root input initializers: 1
2016-05-17 20:45:57,755 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 20:45:57,755 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848]
2016-05-17 20:45:57,758 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 20:45:57,758 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848]
2016-05-17 20:45:57,758 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848]
2016-05-17 20:45:57,758 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848]
2016-05-17 20:45:57,758 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=A4B7AEA4155F4E6FA46F24EE0D2D08B9, vertexId=vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 20:45:57,759 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=A4B7AEA4155F4E6FA46F24EE0D2D08B9, vertexId=vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848]
2016-05-17 20:45:57,761 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=A4B7AEA4155F4E6FA46F24EE0D2D08B9, vertexName=D32E1B14CEFD419D9F85225A98723848, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 20:45:57,762 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=A4B7AEA4155F4E6FA46F24EE0D2D08B9, vertexId=vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848]
2016-05-17 20:45:57,761 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: E9E5621A47A145F1B219FD77E7B7324E on vertex vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4]
2016-05-17 20:45:57,765 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:57,765 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D32E1B14CEFD419D9F85225A98723848, vertexId=vertex_1463517957583_0001_1_01, initRequestedTime=1463517957755, initedTime=1463517957763, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 20:45:57,765 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848] transitioned from NEW to INITED due to event V_INIT
2016-05-17 20:45:57,766 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 20:45:57,767 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input E9E5621A47A145F1B219FD77E7B7324E asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 20:45:57,767 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 20:45:57,767 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 20:45:57,768 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 20:45:57,769 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: E9E5621A47A145F1B219FD77E7B7324E on vertex vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4]
2016-05-17 20:45:57,769 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:57,769 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {E9E5621A47A145F1B219FD77E7B7324E=forAllWorkUnits=true, update=[1]}
2016-05-17 20:45:57,769 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4] parallelism set to 1
2016-05-17 20:45:57,769 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 20:45:57,770 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517957583_0001_1_00, eventCount=1
2016-05-17 20:45:57,770 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 20:45:57,770 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463517957583_0001, with dag id: dag_1463517957583_0001_1
2016-05-17 20:45:57,771 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CE088C9F6B3E48728F84F5607D3748C4, vertexId=vertex_1463517957583_0001_1_00, initRequestedTime=1463517957753, initedTime=1463517957770, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 20:45:57,772 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4] with distanceFromRoot: 0
2016-05-17 20:45:57,772 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 20:45:57,772 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CE088C9F6B3E48728F84F5607D3748C4
2016-05-17 20:45:57,773 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4]
2016-05-17 20:45:57,773 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:57,773 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517957583_0001_1_00, startRequestedTime=1463517957772, startedTime=1463517957772
2016-05-17 20:45:57,774 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:57,775 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:57,775 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:TASK_STARTED]: vertexName=CE088C9F6B3E48728F84F5607D3748C4, taskId=task_1463517957583_0001_1_00_000000, scheduledTime=1463517957775, launchTime=1463517957775
2016-05-17 20:45:57,775 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517957583_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:57,776 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463517957583_0001_1_00 for vertex: vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848] numStartedSources: 1 numSources: 1
2016-05-17 20:45:57,776 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848] with distanceFromRoot: 1
2016-05-17 20:45:57,777 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4]
2016-05-17 20:45:57,778 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: CE088C9F6B3E48728F84F5607D3748C4 in vertex: D32E1B14CEFD419D9F85225A98723848
2016-05-17 20:45:57,778 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: D32E1B14CEFD419D9F85225A98723848 with 1 source tasks and 1 pending tasks
2016-05-17 20:45:57,778 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 20:45:57,779 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463517957583_0001_1_01, startRequestedTime=1463517957777, startedTime=1463517957777
2016-05-17 20:45:57,779 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848] transitioned from INITED to RUNNING due to event V_START
2016-05-17 20:45:57,779 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E] test/(1/1) ...latform/testSimple/simple, VertexName: CE088C9F6B3E48728F84F5607D3748C4, VertexParallelism: 1, TaskAttemptID:attempt_1463517957583_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E9E5621A47A145F1B219FD77E7B7324E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D32E1B14CEFD419D9F85225A98723848, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:57,779 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517957583_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:57,779 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:57,780 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 20:45:57,780 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 20:45:57,781 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 20:45:57,781 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517957583_0001_00_000001
2016-05-17 20:45:57,781 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957583_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:57,783 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517957583_0001_00_000001
2016-05-17 20:45:57,783 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 20:45:57,784 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957583_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:57,784 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:57,784 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517957583_0001_00_000001, launchTime=1463517957784
2016-05-17 20:45:57,785 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517957583_0001_00_000001
2016-05-17 20:45:57,785 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517957583_0001_1_00_000000_0] to container: [container_1463517957583_0001_00_000001]
2016-05-17 20:45:57,786 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957583_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:57,786 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517957583_0001_1_00_000000_0] started. Is using containerId: [container_1463517957583_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 20:45:57,786 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CE088C9F6B3E48728F84F5607D3748C4, taskAttemptId=attempt_1463517957583_0001_1_00_000000_0, startTime=1463517957786, containerId=container_1463517957583_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517957583_0001_00_000001/teamcity, completedLogs=
2016-05-17 20:45:57,786 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517957583_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:57,786 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:57,787 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517957583_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:57,787 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517957583_0001_00_000001 given task: attempt_1463517957583_0001_1_00_000000_0
2016-05-17 20:45:57,787 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517957583_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517957583_0001_1_00_000000_0
2016-05-17 20:45:57 Starting to run new task attempt: attempt_1463517957583_0001_1_00_000000_0
2016-05-17 20:45:57,788 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517957583_0001_1_00_000000_0
2016-05-17 20:45:57,788 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:57,788 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:57,788 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:57,788 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:57,788 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:57,789 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:57,789 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E] test/(1/1) ...latform/testSimple/simple, VertexName: CE088C9F6B3E48728F84F5607D3748C4, VertexParallelism: 1, TaskAttemptID:attempt_1463517957583_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E9E5621A47A145F1B219FD77E7B7324E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D32E1B14CEFD419D9F85225A98723848, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 20:45:57,789 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:57,789 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517957583_0001_1_00_000000_0
2016-05-17 20:45:57,790 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:57,790 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:57,792 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517957583_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 20:45:57,793 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E9E5621A47A145F1B219FD77E7B7324E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 20:45:57,793 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D32E1B14CEFD419D9F85225A98723848, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 20:45:57,793 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:57,794 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:57,794 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:57,795 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E9E5621A47A145F1B219FD77E7B7324E
2016-05-17 20:45:57,795 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: E9E5621A47A145F1B219FD77E7B7324E
2016-05-17 20:45:57,795 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D32E1B14CEFD419D9F85225A98723848
2016-05-17 20:45:57,797 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 20:45:57,797 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: E9E5621A47A145F1B219FD77E7B7324E
2016-05-17 20:45:57,798 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E9E5621A47A145F1B219FD77E7B7324E
2016-05-17 20:45:57,798 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 20:45:57,798 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:57,798 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: D32E1B14CEFD419D9F85225A98723848
2016-05-17 20:45:57,799 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D32E1B14CEFD419D9F85225A98723848
2016-05-17 20:45:57,805 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CE088C9F6B3E48728F84F5607D3748C4, ordinal: 0
2016-05-17 20:45:57,805 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CE088C9F6B3E48728F84F5607D3748C4, mem on start (mb), free: 371, total: 655, max: 672
2016-05-17 20:45:57,805 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:57,805 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:57,805 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:57,807 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:57,807 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:57,807 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 20:45:57,808 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:57,808 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 20:45:57,809 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:57,809 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 20:45:57,809 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E9E5621A47A145F1B219FD77E7B7324E, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 20:45:57,810 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D32E1B14CEFD419D9F85225A98723848, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 20:45:57,810 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:57,810 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E9E5621A47A145F1B219FD77E7B7324E
2016-05-17 20:45:57,810 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:57,811 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E9E5621A47A145F1B219FD77E7B7324E
2016-05-17 20:45:57,811 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E9E5621A47A145F1B219FD77E7B7324E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:57,811 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:57,812 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:57,812 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:57,812 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517957583_0001_1_00_000000_0
2016-05-17 20:45:57,812 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 20:45:57,813 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@4b28f983
2016-05-17 20:45:57,814 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@4b28f983
2016-05-17 20:45:57,814 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 20:45:57,814 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 20:45:57,816 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:57,817 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: E9E5621A47A145F1B219FD77E7B7324E
2016-05-17 20:45:57,817 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: AF0DE59E97534EF6B52278B068B16535
2016-05-17 20:45:57,817 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] AF0DE59E97534EF6B52278B068B16535
2016-05-17 20:45:57,820 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@3defad77; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@42657f65; comparator=cascading.tuple.hadoop.util.TupleComparator@5793a045; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 20:45:57,820 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 20:45:57,821 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 20:45:57,821 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:57,835 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 20:45:57,836 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 20:45:57,836 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 20:45:57,836 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 20:45:57,837 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 20:45:57,837 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 20:45:57,838 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 20:45:57,838 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CE088C9F6B3E48728F84F5607D3748C4, all 1 inputs ready in: 00:00:00.000
2016-05-17 20:45:57,840 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517957583_0001_1_00_000000_0
2016-05-17 20:45:57,841 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CE088C9F6B3E48728F84F5607D3748C4, mem on close (mb), free: 269, total: 655, max: 672
2016-05-17 20:45:57,841 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 20:45:57,841 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 20:45:57,841 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 210; bufvoid = 104857600
2016-05-17 20:45:57,842 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214360(104857440); length = 37/6553600
2016-05-17 20:45:57,844 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 20:45:57 Completed running task attempt: attempt_1463517957583_0001_1_00_000000_0
2016-05-17 20:45:57,848 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517957583_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:57,849 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463517957583_0001_1_00, eventCount=1
2016-05-17 20:45:57,849 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:57,850 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CE088C9F6B3E48728F84F5607D3748C4, taskAttemptId=attempt_1463517957583_0001_1_00_000000_0, startTime=1463517957786, finishTime=1463517957849, timeTaken=63, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686817280, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517957838, Process_Duration=2, Process_End_Time=1463517957840, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 20:45:57,850 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517957583_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:57,850 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:57,850 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517957583_0001_1_00_000000_0
2016-05-17 20:45:57,850 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:57,851 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:TASK_FINISHED]: vertexName=CE088C9F6B3E48728F84F5607D3748C4, taskId=task_1463517957583_0001_1_00_000000, startTime=1463517957786, finishTime=1463517957850, timeTaken=64, status=SUCCEEDED, successfulAttemptID=attempt_1463517957583_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686817280, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517957838, Process_Duration=2, Process_End_Time=1463517957840, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 20:45:57,851 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517957583_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:57,851 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4] : 1
2016-05-17 20:45:57,851 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4]
2016-05-17 20:45:57,851 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686817280, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463517957838, Process_Duration=2, Process_End_Time=1463517957840, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 20:45:57,853 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:VERTEX_FINISHED]: vertexName=CE088C9F6B3E48728F84F5607D3748C4, vertexId=vertex_1463517957583_0001_1_00, initRequestedTime=1463517957753, initedTime=1463517957770, startRequestedTime=1463517957772, startedTime=1463517957772, finishTime=1463517957851, timeTaken=79, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686817280, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517957838, Process_Duration=2, Process_End_Time=1463517957840, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463517957786, firstTasksToStart=[ task_1463517957583_0001_1_00_000000 ], lastTaskFinishTime=1463517957849, lastTasksToFinish=[ task_1463517957583_0001_1_00_000000 ], minTaskDuration=63, maxTaskDuration=63, avgTaskDuration=63.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517957583_0001_1_00_000000 ], longestDurationTasks=[ task_1463517957583_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:57,853 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:57,855 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:57,854 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:57,855 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:57,855 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517957583_0001_00_000001
2016-05-17 20:45:57,855 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957583_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:57,856 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517957583_0001_00_000001, stoppedTime=1463517957856, exitStatus=0
2016-05-17 20:45:57,856 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957583_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:57,856 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848] attempt: attempt_1463517957583_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 20:45:57,856 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D32E1B14CEFD419D9F85225A98723848
2016-05-17 20:45:57,857 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848]
2016-05-17 20:45:57,857 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:57,857 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463517957583_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 20:45:57,856 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:57,858 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 20:45:57,857 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:TASK_STARTED]: vertexName=D32E1B14CEFD419D9F85225A98723848, taskId=task_1463517957583_0001_1_01_000000, scheduledTime=1463517957857, launchTime=1463517957857
2016-05-17 20:45:57,858 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517957583_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 20:45:57,856 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(221)) - No more tasks for container with id : container_1463517957583_0001_00_000001. Asking it to die
2016-05-17 20:45:57,858 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517957583_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:57,858 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517957583_0001_00_000001, Exiting
2016-05-17 20:45:57,859 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517957583_0001_00_000001
2016-05-17 20:45:57,859 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517957583_0001_00_000001
2016-05-17 20:45:57,858 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848]
2016-05-17 20:45:57,859 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957583_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 20:45:57,860 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E] test/(1/1) ...latform/testSimple/simple, VertexName: D32E1B14CEFD419D9F85225A98723848, VertexParallelism: 1, TaskAttemptID:attempt_1463517957583_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CE088C9F6B3E48728F84F5607D3748C4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A4B7AEA4155F4E6FA46F24EE0D2D08B9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:57,860 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517957583_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 20:45:57,860 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 20:45:57,861 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463517957583_0001_00_000002
2016-05-17 20:45:57,861 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957583_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 20:45:57,861 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517957583_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:57,862 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517957583_0001_00_000001 completed successfully
2016-05-17 20:45:57,862 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463517957583_0001_00_000002
2016-05-17 20:45:57,863 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957583_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 20:45:57,863 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 20:45:57,865 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957583_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 20:45:57,865 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 20:45:57,865 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463517957583_0001_00_000002, launchTime=1463517957865
2016-05-17 20:45:57,866 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517957583_0001_00_000002
2016-05-17 20:45:57,866 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463517957583_0001_1_01_000000_0] to container: [container_1463517957583_0001_00_000002]
2016-05-17 20:45:57,866 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957583_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 20:45:57,867 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463517957583_0001_00_000002 given task: attempt_1463517957583_0001_1_01_000000_0
2016-05-17 20:45:57,867 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517957583_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463517957583_0001_1_01_000000_0
2016-05-17 20:45:57,867 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463517957583_0001_1_01_000000_0] started. Is using containerId: [container_1463517957583_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 20:45:57,867 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D32E1B14CEFD419D9F85225A98723848, taskAttemptId=attempt_1463517957583_0001_1_01_000000_0, startTime=1463517957867, containerId=container_1463517957583_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463517957583_0001_00_000002/teamcity, completedLogs=
2016-05-17 20:45:57,868 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517957583_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 20:45:57,868 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517957583_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 20:45:57,868 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 20:45:57 Starting to run new task attempt: attempt_1463517957583_0001_1_01_000000_0
2016-05-17 20:45:57,869 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463517957583_0001_1_01_000000_0
2016-05-17 20:45:57,869 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:57,870 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 20:45:57,870 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 20:45:57,870 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 20:45:57,870 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 20:45:57,870 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 20:45:57,870 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E] test/(1/1) ...latform/testSimple/simple, VertexName: D32E1B14CEFD419D9F85225A98723848, VertexParallelism: 1, TaskAttemptID:attempt_1463517957583_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CE088C9F6B3E48728F84F5607D3748C4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A4B7AEA4155F4E6FA46F24EE0D2D08B9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 20:45:57,870 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 20:45:57,871 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463517957583_0001_1_01_000000_0
2016-05-17 20:45:57,871 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 20:45:57,871 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:57,872 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=CE088C9F6B3E48728F84F5607D3748C4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 20:45:57,872 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 20:45:57,872 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: CE088C9F6B3E48728F84F5607D3748C4
2016-05-17 20:45:57,874 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 20:45:57,874 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 20:45:57,875 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: CE088C9F6B3E48728F84F5607D3748C4
2016-05-17 20:45:57,875 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:57,875 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: CE088C9F6B3E48728F84F5607D3748C4
2016-05-17 20:45:57,876 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463517957583_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 20:45:57,878 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 20:45:57,880 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A4B7AEA4155F4E6FA46F24EE0D2D08B9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 20:45:57,880 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 20:45:57,882 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A4B7AEA4155F4E6FA46F24EE0D2D08B9
2016-05-17 20:45:57,882 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 20:45:57,882 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: A4B7AEA4155F4E6FA46F24EE0D2D08B9
2016-05-17 20:45:57,892 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D32E1B14CEFD419D9F85225A98723848, ordinal: 1
2016-05-17 20:45:57,892 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D32E1B14CEFD419D9F85225A98723848, mem on start (mb), free: 263, total: 655, max: 672
2016-05-17 20:45:57,892 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 20:45:57,893 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 20:45:57,893 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 20:45:57,895 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 20:45:57,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A4B7AEA4155F4E6FA46F24EE0D2D08B9
2016-05-17 20:45:57,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 20:45:57,895 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 20:45:57,895 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 20:45:57,895 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 20:45:57,896 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 20:45:57,896 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 20:45:57,896 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 20:45:57,896 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, CE088C9F6B3E48728F84F5607D3748C4, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 20:45:57,896 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A4B7AEA4155F4E6FA46F24EE0D2D08B9, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 20:45:57,896 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 20:45:57,896 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: CE088C9F6B3E48728F84F5607D3748C4 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 20:45:57,896 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: CE088C9F6B3E48728F84F5607D3748C4
2016-05-17 20:45:57,897 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 20:45:57,897 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 20:45:57,897 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 20:45:57,897 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 20:45:57,898 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: CE088C9F6B3E48728F84F5607D3748C4 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 20:45:57,898 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 20:45:57,898 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 20:45:57,899 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 20:45:57,899 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 20:45:57,900 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 20:45:57,901 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: CE088C9F6B3E48728F84F5607D3748C4
2016-05-17 20:45:57,906 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 20:45:57,906 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463517957583_0001_1_01_000000_0
2016-05-17 20:45:57,908 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463517957583_0001_1_00_000000_0_10089, runDuration: 50564, ]
2016-05-17 20:45:57,908 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-05-17 20:45:57,909 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: AF0DE59E97534EF6B52278B068B16535
2016-05-17 20:45:57,909 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463517957583_0001_1_00_000000_0_10089] to DISK_DIRECT, CompressedSize=204, DecompressedSize=200,EndTime=1463517957909, TimeTaken=1, Rate=0.19 MB/s
2016-05-17 20:45:57,909 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple"], id: A4B7AEA4155F4E6FA46F24EE0D2D08B9
2016-05-17 20:45:57,909 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : CE088C9F6B3E48728F84F5607D3748C4
2016-05-17 20:45:57,909 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple"]
2016-05-17 20:45:57,910 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 20:45:57,910 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] AF0DE59E97534EF6B52278B068B16535, for 1 inputs
2016-05-17 20:45:57,910 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [CE088C9F6B3E48728F84F5607D3748C4] #14 in 2ms
2016-05-17 20:45:57,910 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [CE088C9F6B3E48728F84F5607D3748C4] #14, status:false, isInterrupted:false
2016-05-17 20:45:57,912 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 20:45:57,913 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 204 bytes from disk
2016-05-17 20:45:57,913 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:57,913 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 20:45:57,914 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 175 bytes
2016-05-17 20:45:57,914 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D32E1B14CEFD419D9F85225A98723848, all 1 inputs ready in: 00:00:00.004
2016-05-17 20:45:57,915 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : CE088C9F6B3E48728F84F5607D3748C4
2016-05-17 20:45:57,916 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@31323d55; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@5958b0ae
2016-05-17 20:45:57,916 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 20:45:57,919 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 20:45:57,919 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 20:45:57,920 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463517957583_0001_1_01_000000_0
2016-05-17 20:45:57,920 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463517957583_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 20:45:57,921 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463517957583_0001_1_01_000000_0
2016-05-17 20:45:57,921 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D32E1B14CEFD419D9F85225A98723848, mem on close (mb), free: 261, total: 655, max: 672
2016-05-17 20:45:57,922 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: CE088C9F6B3E48728F84F5607D3748C4
2016-05-17 20:45:57,922 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 20:45:57,923 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 20:45:57,923 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 20:45:57,923 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 20:45:57,923 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 20:45:57 Completed running task attempt: attempt_1463517957583_0001_1_01_000000_0
2016-05-17 20:45:57,925 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463517957583_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 20:45:57,926 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686817280, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463517957914, Process_Duration=7, Process_End_Time=1463517957921, Read_Duration=1, Tuples_Read=10, Tuples_Written=8, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=8]]
2016-05-17 20:45:57,926 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D32E1B14CEFD419D9F85225A98723848, taskAttemptId=attempt_1463517957583_0001_1_01_000000_0, startTime=1463517957867, finishTime=1463517957926, timeTaken=59, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686817280, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517957914, Process_Duration=7, Process_End_Time=1463517957921, Read_Duration=1, Tuples_Read=10, Tuples_Written=8, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=8
2016-05-17 20:45:57,926 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 20:45:57,926 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 20:45:57,926 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:57,926 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463517957583_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 20:45:57,927 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463517957583_0001_1_01_000000_0
2016-05-17 20:45:57,927 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 20:45:57,927 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 20:45:57,928 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:TASK_FINISHED]: vertexName=D32E1B14CEFD419D9F85225A98723848, taskId=task_1463517957583_0001_1_01_000000, startTime=1463517957867, finishTime=1463517957927, timeTaken=60, status=SUCCEEDED, successfulAttemptID=attempt_1463517957583_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686817280, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517957914, Process_Duration=7, Process_End_Time=1463517957921, Read_Duration=1, Tuples_Read=10, Tuples_Written=8, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=8
2016-05-17 20:45:57,928 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463517957583_0001_00_000002
2016-05-17 20:45:57,928 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463517957583_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 20:45:57,928 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848] : 1
2016-05-17 20:45:57,928 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848]
2016-05-17 20:45:57,929 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 20:45:57,929 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:VERTEX_FINISHED]: vertexName=D32E1B14CEFD419D9F85225A98723848, vertexId=vertex_1463517957583_0001_1_01, initRequestedTime=1463517957755, initedTime=1463517957763, startRequestedTime=1463517957777, startedTime=1463517957777, finishTime=1463517957928, timeTaken=151, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=686817280, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463517957914, Process_Duration=7, Process_End_Time=1463517957921, Read_Duration=1, Tuples_Read=10, Tuples_Written=8, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=8, vertexStats=firstTaskStartTime=1463517957867, firstTasksToStart=[ task_1463517957583_0001_1_01_000000 ], lastTaskFinishTime=1463517957926, lastTasksToFinish=[ task_1463517957583_0001_1_01_000000 ], minTaskDuration=59, maxTaskDuration=59, avgTaskDuration=59.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463517957583_0001_1_01_000000 ], longestDurationTasks=[ task_1463517957583_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 20:45:57,929 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 20:45:57,930 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957583_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 20:45:57,930 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463517957583_0001_00_000002, stoppedTime=1463517957930, exitStatus=0
2016-05-17 20:45:57,930 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 20:45:57,930 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957583_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 20:45:57,931 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 20:45:57,931 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463517957583_0001_1
2016-05-17 20:45:57,931 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463517957583_0001_1
2016-05-17 20:45:57,931 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: A4B7AEA4155F4E6FA46F24EE0D2D08B9 for vertex: vertex_1463517957583_0001_1_01 [D32E1B14CEFD419D9F85225A98723848]
2016-05-17 20:45:57,939 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463517957583_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 20:45:57,939 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463517957583_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 20:45:57,939 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463517957583_0001_00_000002, Exiting
2016-05-17 20:45:57,939 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463517957583_0001_00_000002
2016-05-17 20:45:57,939 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463517957583_0001_00_000002
2016-05-17 20:45:57,939 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463517957583_0001_1_00 [CE088C9F6B3E48728F84F5607D3748C4]
2016-05-17 20:45:57,939 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463517957583_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 20:45:57,940 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463517957583_0001_00_000002 completed successfully
2016-05-17 20:45:57,942 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463517957583_0001_1, queueSize=0
2016-05-17 20:45:57,942 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 20:45:57,943 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463517957583_0001_1][Event:DAG_FINISHED]: dagId=dag_1463517957583_0001_1, startTime=1463517957752, finishTime=1463517957940, timeTaken=188, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2442, FILE_BYTES_WRITTEN=376, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1373634560, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=18, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927035915752, Process_Duration=9, Process_End_Time=2927035915761, Read_Duration=3, Tuples_Read=20, Tuples_Written=18, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=8
2016-05-17 20:45:57,943 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463517957583_0001_1 finished with state: SUCCEEDED
2016-05-17 20:45:57,943 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463517957583_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 20:45:57,943 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463517957583_0001_00_000002
2016-05-17 20:45:57,944 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463517957583_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 20:45:57 Completed Dag: dag_1463517957583_0001_1
2016-05-17 20:45:57,945 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463517957583_0001_1, dagState=SUCCEEDED
2016-05-17 20:45:57,946 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463517957583_0001_1_post
2016-05-17 20:45:57,947 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 20:45:57,947 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 20:45:57,951 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testSimple/simple, applicationId=application_1463517957583_0001
2016-05-17 20:45:57,953 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 20:45:57,953 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 20:45:57,953 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 20:45:57,953 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 20:45:57,953 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 20:45:57,954 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 20:45:57,955 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 20:45:57,955 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 20:45:57,955 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 20:45:57,955 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 20:45:57,956 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 20:45:57,956 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 20:45:57,956 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 20:45:57,961 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D302E8C4373F44E6A5EE3A0DAE43F1C1/C80A10920C574750AA1E380C7D0D918E/.tez/application_1463517957583_0001
2016-05-17 20:45:57,961 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple/_temporary
2016-05-17 20:45:57,962 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.398
2016-05-17 20:45:57,993 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimple/simple