2016-01-14 15:21:38,486 INFO platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(195)) - found 1 test platforms from classpath
2016-01-14 15:21:38,508 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(244)) - adding test: cascading.BasicPipesPlatformTest, with platform: hadoop2-tez
2016-01-14 15:21:38,535 INFO tez.Hadoop2TezPlatform (Hadoop2TezPlatform.java:setUp(136)) - not using cluster
2016-01-14 15:21:38,900 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-01-14 15:21:39,267 INFO platform.TestPlatform (TestPlatform.java:getGlobalProperties(73)) - platform property overrides:
2016-01-14 15:21:39,525 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:21:39,528 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:21:39,648 INFO property.AppProps (AppProps.java:getAppID(177)) - using app.id: 49426FA6EE1B4ACDAB0CC7398723D58E
2016-01-14 15:21:39,891 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.108
2016-01-14 15:21:39,930 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.147
2016-01-14 15:21:39,932 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-01-14 15:21:39,932 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-01-14 15:21:39,934 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:21:40,098 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-01-14 15:21:40,281 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain
2016-01-14 15:21:40,539 INFO util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 3.0.4
2016-01-14 15:21:40,543 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 15:21:40,544 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 15:21:40,546 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain"]
2016-01-14 15:21:40,546 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 15:21:40,546 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 15:21:40,547 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 15:21:40,550 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...stSimpleChain/simplechain
2016-01-14 15:21:40,550 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:21:40,567 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 15:21:40,741 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:40,750 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:21:40,750 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:21:40,807 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D1D5FAED70EA45E9BB5FD560F8143B83/BED9C9F811E94D3E810B993D1E951BB7/.tez/application_1452784900749_0001 doesn't exist and is created
2016-01-14 15:21:40,876 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:21:40,876 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:21:40,877 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D1D5FAED70EA45E9BB5FD560F8143B83/BED9C9F811E94D3E810B993D1E951BB7/.tez/application_1452784900749_0001
2016-01-14 15:21:40,915 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452784900749_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:40,955 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:21:40,970 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:21:40,977 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: NEW
2016-01-14 15:21:40,977 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(238)) - DAGAppMaster is not started wait for 100ms...
2016-01-14 15:21:40,977 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:21:40,997 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:21:41,001 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:21:41,005 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D1D5FAED70EA45E9BB5FD560F8143B83/BED9C9F811E94D3E810B993D1E951BB7/.tez/application_1452784900749_0001/localmode-log-dir
2016-01-14 15:21:41,006 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D1D5FAED70EA45E9BB5FD560F8143B83/BED9C9F811E94D3E810B993D1E951BB7/.tez/application_1452784900749_0001/localmode-log-dir/history.txt.appattempt_1452784900749_0001_000000, maxErrors=10
2016-01-14 15:21:41,007 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:21:41,009 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452784900749_0001_000000, appSubmitTime=1452784900897, launchTime=1452784900914
2016-01-14 15:21:41,042 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:21:41,042 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:21:41,047 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:21:41,049 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:21:41,077 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-01-14 15:21:41,078 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-01-14 15:21:41,078 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:21:41,113 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:56756
2016-01-14 15:21:41,151 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:21:41,178 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-01-14 15:21:41,178 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-01-14 15:21:41,178 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:21:41,223 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:21:41,224 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452784900749_0001_000000, startTime=1452784901222
2016-01-14 15:21:41,224 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:21:41,279 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:21:41,294 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:21:41,294 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...stSimpleChain/simplechain, applicationId=application_1452784900749_0001, dagName=[D1D5FAED70EA45E9BB5FD560F8143B83/BED9C9F811E94D3E810B993D1E951BB7] test/(1/1) ...stSimpleChain/simplechain
2016-01-14 15:21:41,499 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [D1D5FAED70EA45E9BB5FD560F8143B83/BED9C9F811E94D3E810B993D1E951BB7] test/(1/1) ...stSimpleChain/simplechain
2016-01-14 15:21:41,599 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452784900749_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D1D5FAED70EA45E9BB5FD560F8143B83/BED9C9F811E94D3E810B993D1E951BB7/.tez/application_1452784900749_0001/localmode-log-dir/dag_1452784900749_0001_1.dot
2016-01-14 15:21:41,604 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D1D5FAED70EA45E9BB5FD560F8143B83/BED9C9F811E94D3E810B993D1E951BB7/.tez/application_1452784900749_0001/localmode-log-dir/dag_1452784900749_0001_1-tez-dag.pb.txt
2016-01-14 15:21:41,696 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784900749_0001_1
2016-01-14 15:21:41,696 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:41,697 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [D1D5FAED70EA45E9BB5FD560F8143B83/BED9C9F811E94D3E810B993D1E951BB7] test/(1/1) ...stSimpleChain/simplechain
2016-01-14 15:21:41 Running Dag: dag_1452784900749_0001_1
2016-01-14 15:21:41,736 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:21:41,738 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452784900749_0001_1, submitTime=1452784901500
2016-01-14 15:21:41,775 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DA881B27594E4E0EBAF60521FB2E6C42
2016-01-14 15:21:41,783 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:21:41,786 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:21:41,786 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452784900749_0001_1, initTime=1452784901740
2016-01-14 15:21:41,788 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784900749_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:21:41,790 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:21:41,790 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:DAG_STARTED]: dagID=dag_1452784900749_0001_1, startTime=1452784901789
2016-01-14 15:21:41,791 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:21:41,794 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784900749_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:21:41,795 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BE65D4E3697E4277ACD95B3D75CC8F23 : {B01599F28018438497C9B39916421328={InputName=B01599F28018438497C9B39916421328}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:21:41,795 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: B01599F28018438497C9B39916421328, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:21:41,795 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452784900749_0001_1_00 [BE65D4E3697E4277ACD95B3D75CC8F23]
2016-01-14 15:21:41,800 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...stSimpleChain/simplechain, applicationId=application_1452784900749_0001, dagName=[D1D5FAED70EA45E9BB5FD560F8143B83/BED9C9F811E94D3E810B993D1E951BB7] test/(1/1) ...stSimpleChain/simplechain
2016-01-14 15:21:41,803 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452784900749_0001_1_00 [BE65D4E3697E4277ACD95B3D75CC8F23]
2016-01-14 15:21:41,803 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452784900749_0001_1_00 [BE65D4E3697E4277ACD95B3D75CC8F23]
2016-01-14 15:21:41,806 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452784900749_0001_1_00 [BE65D4E3697E4277ACD95B3D75CC8F23]. Starting root input initializers: 1
2016-01-14 15:21:41,821 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: B01599F28018438497C9B39916421328 on vertex vertex_1452784900749_0001_1_00 [BE65D4E3697E4277ACD95B3D75CC8F23]
2016-01-14 15:21:41,831 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452784900749_0001, with dag id: dag_1452784900749_0001_1
2016-01-14 15:21:41,832 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784900749_0001_1_00 [BE65D4E3697E4277ACD95B3D75CC8F23] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:21:41,836 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452784900749_0001_1_01 [A9A1ADFCDDDF4A0E8658ED305245FDD4]
2016-01-14 15:21:41,860 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:21:41,863 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input B01599F28018438497C9B39916421328 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:21:41,864 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:21:41,871 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:21:41,876 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:21:41,876 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452784900749_0001_1_01 [A9A1ADFCDDDF4A0E8658ED305245FDD4]
2016-01-14 15:21:41,889 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452784900749_0001_1_01 [A9A1ADFCDDDF4A0E8658ED305245FDD4]
2016-01-14 15:21:41,892 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:41,892 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A9A1ADFCDDDF4A0E8658ED305245FDD4, vertexId=vertex_1452784900749_0001_1_01, initRequestedTime=1452784901834, initedTime=1452784901891, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:21:41,893 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784900749_0001_1_01 [A9A1ADFCDDDF4A0E8658ED305245FDD4] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:21:41,894 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452784900749_0001_1_02 [DA881B27594E4E0EBAF60521FB2E6C42]
2016-01-14 15:21:41,953 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:21:41,953 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-01-14 15:21:41,954 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452784900749_0001_1_02 [DA881B27594E4E0EBAF60521FB2E6C42]
2016-01-14 15:21:41,956 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452784900749_0001_1_02 [DA881B27594E4E0EBAF60521FB2E6C42]
2016-01-14 15:21:41,957 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452784900749_0001_1_02 [DA881B27594E4E0EBAF60521FB2E6C42]
2016-01-14 15:21:41,957 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=4A465710519144119168DF5824F2FB92, vertexId=vertex_1452784900749_0001_1_02 [DA881B27594E4E0EBAF60521FB2E6C42], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:21:41,959 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: B01599F28018438497C9B39916421328 on vertex vertex_1452784900749_0001_1_00 [BE65D4E3697E4277ACD95B3D75CC8F23]
2016-01-14 15:21:41,961 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=4A465710519144119168DF5824F2FB92, vertexId=vertex_1452784900749_0001_1_02 [DA881B27594E4E0EBAF60521FB2E6C42]
2016-01-14 15:21:41,977 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=4A465710519144119168DF5824F2FB92, vertexName=DA881B27594E4E0EBAF60521FB2E6C42, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:21:41,988 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=4A465710519144119168DF5824F2FB92, vertexId=vertex_1452784900749_0001_1_02 [DA881B27594E4E0EBAF60521FB2E6C42]
2016-01-14 15:21:41,991 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DA881B27594E4E0EBAF60521FB2E6C42, vertexId=vertex_1452784900749_0001_1_02, initRequestedTime=1452784901894, initedTime=1452784901991, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:21:41,991 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:41,992 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784900749_0001_1_02 [DA881B27594E4E0EBAF60521FB2E6C42] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:21:41,993 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784900749_0001_1_00 [BE65D4E3697E4277ACD95B3D75CC8F23] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:41,993 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {B01599F28018438497C9B39916421328=forAllWorkUnits=true, update=[1]}
2016-01-14 15:21:41,994 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452784900749_0001_1_00 [BE65D4E3697E4277ACD95B3D75CC8F23] parallelism set to 1
2016-01-14 15:21:41,994 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784900749_0001_1_00 [BE65D4E3697E4277ACD95B3D75CC8F23] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:42,003 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784900749_0001_1_00, eventCount=1
2016-01-14 15:21:42,004 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BE65D4E3697E4277ACD95B3D75CC8F23, vertexId=vertex_1452784900749_0001_1_00, initRequestedTime=1452784901794, initedTime=1452784902004, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:21:42,005 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784900749_0001_1_00 [BE65D4E3697E4277ACD95B3D75CC8F23] with distanceFromRoot: 0
2016-01-14 15:21:42,005 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784900749_0001_1_00 [BE65D4E3697E4277ACD95B3D75CC8F23] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:21:42,007 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:42,011 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BE65D4E3697E4277ACD95B3D75CC8F23
2016-01-14 15:21:42,012 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784900749_0001_1_00 [BE65D4E3697E4277ACD95B3D75CC8F23]
2016-01-14 15:21:42,015 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:42,015 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784900749_0001_1_00, startRequestedTime=1452784902011, startedTime=1452784902011
2016-01-14 15:21:42,016 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784900749_0001_1_00 [BE65D4E3697E4277ACD95B3D75CC8F23] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:42,040 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:42,040 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:TASK_STARTED]: vertexName=BE65D4E3697E4277ACD95B3D75CC8F23, taskId=task_1452784900749_0001_1_00_000000, scheduledTime=1452784902039, launchTime=1452784902039
2016-01-14 15:21:42,041 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784900749_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:42,041 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452784900749_0001_1_00 for vertex: vertex_1452784900749_0001_1_01 [A9A1ADFCDDDF4A0E8658ED305245FDD4] numStartedSources: 1 numSources: 1
2016-01-14 15:21:42,041 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784900749_0001_1_01 [A9A1ADFCDDDF4A0E8658ED305245FDD4] with distanceFromRoot: 1
2016-01-14 15:21:42,044 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784900749_0001_1_00 [BE65D4E3697E4277ACD95B3D75CC8F23]
2016-01-14 15:21:42,048 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: BE65D4E3697E4277ACD95B3D75CC8F23 in vertex: A9A1ADFCDDDF4A0E8658ED305245FDD4
2016-01-14 15:21:42,051 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A9A1ADFCDDDF4A0E8658ED305245FDD4 with 1 source tasks and 1 pending tasks
2016-01-14 15:21:42,052 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:42,052 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784900749_0001_1_01, startRequestedTime=1452784902045, startedTime=1452784902045
2016-01-14 15:21:42,052 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784900749_0001_1_01 [A9A1ADFCDDDF4A0E8658ED305245FDD4] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:42,059 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D1D5FAED70EA45E9BB5FD560F8143B83/BED9C9F811E94D3E810B993D1E951BB7] test/(1/1) ...stSimpleChain/simplechain, VertexName: BE65D4E3697E4277ACD95B3D75CC8F23, VertexParallelism: 1, TaskAttemptID:attempt_1452784900749_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B01599F28018438497C9B39916421328, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A9A1ADFCDDDF4A0E8658ED305245FDD4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:21:42,061 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784900749_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:42,062 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452784900749_0001_1_01 for vertex: vertex_1452784900749_0001_1_02 [DA881B27594E4E0EBAF60521FB2E6C42] numStartedSources: 1 numSources: 1
2016-01-14 15:21:42,062 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784900749_0001_1_02 [DA881B27594E4E0EBAF60521FB2E6C42] with distanceFromRoot: 2
2016-01-14 15:21:42,062 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:42,063 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: A9A1ADFCDDDF4A0E8658ED305245FDD4 in vertex: DA881B27594E4E0EBAF60521FB2E6C42
2016-01-14 15:21:42,063 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: DA881B27594E4E0EBAF60521FB2E6C42 with 1 source tasks and 1 pending tasks
2016-01-14 15:21:42,064 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:42,064 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784900749_0001_1_02, startRequestedTime=1452784902062, startedTime=1452784902062
2016-01-14 15:21:42,064 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784900749_0001_1_02 [DA881B27594E4E0EBAF60521FB2E6C42] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:42,105 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:21:42,109 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:21:42,109 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:21:42,113 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784900749_0001_00_000001
2016-01-14 15:21:42,114 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784900749_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:42,123 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784900749_0001_00_000001
2016-01-14 15:21:42,123 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:21:42,128 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:42,129 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784900749_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:42,129 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784900749_0001_00_000001, launchTime=1452784902128
2016-01-14 15:21:42,132 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784900749_0001_00_000001
2016-01-14 15:21:42,137 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784900749_0001_1_00_000000_0] to container: [container_1452784900749_0001_00_000001]
2016-01-14 15:21:42,137 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784900749_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:42,138 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784900749_0001_00_000001 given task: attempt_1452784900749_0001_1_00_000000_0
2016-01-14 15:21:42,138 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784900749_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784900749_0001_1_00_000000_0
2016-01-14 15:21:42 Starting to run new task attempt: attempt_1452784900749_0001_1_00_000000_0
2016-01-14 15:21:42,139 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784900749_0001_1_00_000000_0
2016-01-14 15:21:42,139 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:42,141 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:42,141 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:42,141 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:42,141 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784900749_0001_1_00_000000_0] started. Is using containerId: [container_1452784900749_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:21:42,143 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:42,143 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:42,143 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BE65D4E3697E4277ACD95B3D75CC8F23, taskAttemptId=attempt_1452784900749_0001_1_00_000000_0, startTime=1452784902138, containerId=container_1452784900749_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784900749_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:21:42,144 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784900749_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:42,145 INFO counters.Limits (Limits.java:ensureInitialized(59)) - Counter limits initialized with parameters: GROUP_NAME_MAX=128, MAX_GROUPS=500, COUNTER_NAME_MAX=64, MAX_COUNTERS=1200
2016-01-14 15:21:42,149 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784900749_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:42,175 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:42,176 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D1D5FAED70EA45E9BB5FD560F8143B83/BED9C9F811E94D3E810B993D1E951BB7] test/(1/1) ...stSimpleChain/simplechain, VertexName: BE65D4E3697E4277ACD95B3D75CC8F23, VertexParallelism: 1, TaskAttemptID:attempt_1452784900749_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B01599F28018438497C9B39916421328, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A9A1ADFCDDDF4A0E8658ED305245FDD4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:21:42,181 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:42,188 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784900749_0001_1_00_000000_0
2016-01-14 15:21:42,188 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:42,189 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:42,192 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784900749_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:21:42,200 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B01599F28018438497C9B39916421328, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:21:42,203 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:42,203 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A9A1ADFCDDDF4A0E8658ED305245FDD4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:21:42,203 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:42,204 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B01599F28018438497C9B39916421328
2016-01-14 15:21:42,205 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:42,210 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: B01599F28018438497C9B39916421328
2016-01-14 15:21:42,214 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A9A1ADFCDDDF4A0E8658ED305245FDD4
2016-01-14 15:21:42,230 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:42,238 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:21:42,239 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A9A1ADFCDDDF4A0E8658ED305245FDD4
2016-01-14 15:21:42,240 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:21:42,241 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A9A1ADFCDDDF4A0E8658ED305245FDD4
2016-01-14 15:21:42,245 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: B01599F28018438497C9B39916421328
2016-01-14 15:21:42,245 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B01599F28018438497C9B39916421328
2016-01-14 15:21:42,263 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BE65D4E3697E4277ACD95B3D75CC8F23, ordinal: 0
2016-01-14 15:21:42,265 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BE65D4E3697E4277ACD95B3D75CC8F23, mem on start (mb), free: 241, total: 281, max: 672
2016-01-14 15:21:42,265 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:42,265 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:42,267 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:42,267 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:42,268 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:42,275 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:21:42,276 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:42,277 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:21:42,278 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:21:42,279 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:21:42,279 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B01599F28018438497C9B39916421328, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:21:42,279 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A9A1ADFCDDDF4A0E8658ED305245FDD4, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:21:42,280 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:42,281 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B01599F28018438497C9B39916421328 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:42,281 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:21:42,282 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:42,282 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B01599F28018438497C9B39916421328
2016-01-14 15:21:42,282 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:42,283 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B01599F28018438497C9B39916421328
2016-01-14 15:21:42,284 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:42,285 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784900749_0001_1_00_000000_0
2016-01-14 15:21:42,287 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:21:42,298 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@5ba8e795
2016-01-14 15:21:42,311 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@5ba8e795
2016-01-14 15:21:42,312 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:21:42,312 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:21:42,324 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 15:21:42,355 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: B01599F28018438497C9B39916421328
2016-01-14 15:21:42,356 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: D50720A172AE4E32A5A403804778ADB5
2016-01-14 15:21:42,356 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] D50720A172AE4E32A5A403804778ADB5
2016-01-14 15:21:42,375 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@5a195c09; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@36e98115; comparator=cascading.tuple.hadoop.util.TupleComparator@1c19a5dd; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:21:42,380 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:21:42,383 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:21:42,383 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:21:42,499 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:21:42,500 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:21:42,500 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:21:42,500 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:21:42,501 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:21:42,508 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 15:21:42,508 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:42,510 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BE65D4E3697E4277ACD95B3D75CC8F23, all 1 inputs ready in: 00:00:00.001
2016-01-14 15:21:42,519 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784900749_0001_1_00_000000_0
2016-01-14 15:21:42,519 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BE65D4E3697E4277ACD95B3D75CC8F23, mem on close (mb), free: 131, total: 281, max: 672
2016-01-14 15:21:42,520 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:21:42,520 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:21:42,521 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 210; bufvoid = 104857600
2016-01-14 15:21:42,521 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214360(104857440); length = 37/6553600
2016-01-14 15:21:42,539 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:21:42 Completed running task attempt: attempt_1452784900749_0001_1_00_000000_0
2016-01-14 15:21:42,558 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784900749_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:42,559 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784900749_0001_1_00, eventCount=1
2016-01-14 15:21:42,561 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=1272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452784902511, Process_Duration=8, Process_End_Time=1452784902519, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10]]
2016-01-14 15:21:42,562 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:42,562 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:42,563 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:42,563 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784900749_0001_00_000001
2016-01-14 15:21:42,567 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BE65D4E3697E4277ACD95B3D75CC8F23, taskAttemptId=attempt_1452784900749_0001_1_00_000000_0, startTime=1452784902138, finishTime=1452784902560, timeTaken=422, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=1272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784902511, Process_Duration=8, Process_End_Time=1452784902519, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-01-14 15:21:42,568 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784900749_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:42,568 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:42,571 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784900749_0001_1_00_000000_0
2016-01-14 15:21:42,574 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:42,578 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:TASK_FINISHED]: vertexName=BE65D4E3697E4277ACD95B3D75CC8F23, taskId=task_1452784900749_0001_1_00_000000, startTime=1452784902138, finishTime=1452784902573, timeTaken=435, status=SUCCEEDED, successfulAttemptID=attempt_1452784900749_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=1272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784902511, Process_Duration=8, Process_End_Time=1452784902519, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-01-14 15:21:42,579 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784900749_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:42,579 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784900749_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:42,581 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784900749_0001_1_00 [BE65D4E3697E4277ACD95B3D75CC8F23] : 1
2016-01-14 15:21:42,581 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784900749_0001_1_00 [BE65D4E3697E4277ACD95B3D75CC8F23]
2016-01-14 15:21:42,597 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:42,601 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:VERTEX_FINISHED]: vertexName=BE65D4E3697E4277ACD95B3D75CC8F23, vertexId=vertex_1452784900749_0001_1_00, initRequestedTime=1452784901794, initedTime=1452784902004, startRequestedTime=1452784902011, startedTime=1452784902011, finishTime=1452784902582, timeTaken=571, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=1272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784902511, Process_Duration=8, Process_End_Time=1452784902519, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1452784902138, firstTasksToStart=[ task_1452784900749_0001_1_00_000000 ], lastTaskFinishTime=1452784902560, lastTasksToFinish=[ task_1452784900749_0001_1_00_000000 ], minTaskDuration=422, maxTaskDuration=422, avgTaskDuration=422.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784900749_0001_1_00_000000 ], longestDurationTasks=[ task_1452784900749_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:42,602 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784900749_0001_1_00 [BE65D4E3697E4277ACD95B3D75CC8F23] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:42,603 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784900749_0001_00_000001, stoppedTime=1452784902603, exitStatus=0
2016-01-14 15:21:42,604 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:42,605 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784900749_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:42,605 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452784900749_0001_1_01 [A9A1ADFCDDDF4A0E8658ED305245FDD4] attempt: attempt_1452784900749_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:21:42,606 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A9A1ADFCDDDF4A0E8658ED305245FDD4
2016-01-14 15:21:42,606 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784900749_0001_1_01 [A9A1ADFCDDDF4A0E8658ED305245FDD4]
2016-01-14 15:21:42,609 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784900749_0001_1_00 [BE65D4E3697E4277ACD95B3D75CC8F23] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 15:21:42,609 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452784900749_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:21:42,610 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:42,610 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:TASK_STARTED]: vertexName=A9A1ADFCDDDF4A0E8658ED305245FDD4, taskId=task_1452784900749_0001_1_01_000000, scheduledTime=1452784902610, launchTime=1452784902610
2016-01-14 15:21:42,611 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784900749_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:42,611 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784900749_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:21:42,612 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784900749_0001_1_01 [A9A1ADFCDDDF4A0E8658ED305245FDD4]
2016-01-14 15:21:42,612 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D1D5FAED70EA45E9BB5FD560F8143B83/BED9C9F811E94D3E810B993D1E951BB7] test/(1/1) ...stSimpleChain/simplechain, VertexName: A9A1ADFCDDDF4A0E8658ED305245FDD4, VertexParallelism: 1, TaskAttemptID:attempt_1452784900749_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BE65D4E3697E4277ACD95B3D75CC8F23, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=DA881B27594E4E0EBAF60521FB2E6C42, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:21:42,613 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784900749_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:42,613 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:42,615 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784900749_0001_00_000002
2016-01-14 15:21:42,615 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784900749_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:42,619 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784900749_0001_00_000002
2016-01-14 15:21:42,619 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:21:42,621 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:42,621 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784900749_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:42,621 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784900749_0001_00_000002, launchTime=1452784902620
2016-01-14 15:21:42,622 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784900749_0001_00_000002
2016-01-14 15:21:42,624 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784900749_0001_1_01_000000_0] to container: [container_1452784900749_0001_00_000002]
2016-01-14 15:21:42,624 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784900749_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:42,625 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784900749_0001_00_000002 given task: attempt_1452784900749_0001_1_01_000000_0
2016-01-14 15:21:42,626 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784900749_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784900749_0001_1_01_000000_0
2016-01-14 15:21:42,625 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784900749_0001_1_01_000000_0] started. Is using containerId: [container_1452784900749_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:21:42,627 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:42,628 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A9A1ADFCDDDF4A0E8658ED305245FDD4, taskAttemptId=attempt_1452784900749_0001_1_01_000000_0, startTime=1452784902625, containerId=container_1452784900749_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784900749_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:21:42,628 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784900749_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:21:42,628 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784900749_0001_00_000001: 63 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:42,629 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784900749_0001_00_000001, Exiting
2016-01-14 15:21:42,629 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784900749_0001_00_000001
2016-01-14 15:21:42,629 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784900749_0001_00_000001
2016-01-14 15:21:42 Starting to run new task attempt: attempt_1452784900749_0001_1_01_000000_0
2016-01-14 15:21:42,629 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784900749_0001_1_01_000000_0
2016-01-14 15:21:42,629 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:42,630 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:42,630 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:42,630 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:42,630 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:42,628 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784900749_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:42,630 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:42,633 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D1D5FAED70EA45E9BB5FD560F8143B83/BED9C9F811E94D3E810B993D1E951BB7] test/(1/1) ...stSimpleChain/simplechain, VertexName: A9A1ADFCDDDF4A0E8658ED305245FDD4, VertexParallelism: 1, TaskAttemptID:attempt_1452784900749_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BE65D4E3697E4277ACD95B3D75CC8F23, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=DA881B27594E4E0EBAF60521FB2E6C42, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:21:42,634 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:42,634 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784900749_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:42,635 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784900749_0001_1_01_000000_0
2016-01-14 15:21:42,636 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:42,636 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:42,637 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784900749_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:42,638 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784900749_0001_00_000001 completed successfully
2016-01-14 15:21:42,640 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784900749_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:21:42,642 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784900749_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:21:42,642 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BE65D4E3697E4277ACD95B3D75CC8F23, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:21:42,644 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:42,644 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DA881B27594E4E0EBAF60521FB2E6C42, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:21:42,645 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:42,645 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DA881B27594E4E0EBAF60521FB2E6C42
2016-01-14 15:21:42,652 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:42,653 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:42,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BE65D4E3697E4277ACD95B3D75CC8F23
2016-01-14 15:21:42,666 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:21:42,667 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: DA881B27594E4E0EBAF60521FB2E6C42
2016-01-14 15:21:42,667 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DA881B27594E4E0EBAF60521FB2E6C42
2016-01-14 15:21:42,676 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A9A1ADFCDDDF4A0E8658ED305245FDD4, ordinal: 1
2016-01-14 15:21:42,676 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:21:42,676 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A9A1ADFCDDDF4A0E8658ED305245FDD4, mem on start (mb), free: 120, total: 281, max: 672
2016-01-14 15:21:42,676 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:42,677 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:42,677 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:42,676 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:21:42,678 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: BE65D4E3697E4277ACD95B3D75CC8F23
2016-01-14 15:21:42,679 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BE65D4E3697E4277ACD95B3D75CC8F23
2016-01-14 15:21:42,679 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:42,679 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:42,681 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:21:42,681 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:21:42,681 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:21:42,682 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 24, TotalRequested: 739036352, TotalRequestedScaled: 3.69518176E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:1.05
2016-01-14 15:21:42,682 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DA881B27594E4E0EBAF60521FB2E6C42, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=66985083
2016-01-14 15:21:42,683 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BE65D4E3697E4277ACD95B3D75CC8F23, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=405125774
2016-01-14 15:21:42,686 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:42,687 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BE65D4E3697E4277ACD95B3D75CC8F23 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:42,687 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BE65D4E3697E4277ACD95B3D75CC8F23
2016-01-14 15:21:42,687 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:21:42,694 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:42,700 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:21:42,701 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:21:42,706 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: BE65D4E3697E4277ACD95B3D75CC8F23 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:21:42,709 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=405125774. Updated to: ShuffleMem=405125774, postMergeMem: 0
2016-01-14 15:21:42,710 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=405125774, maxSingleShuffleLimit=101281440, mergeThreshold=364613184, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:21:42,717 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:21:42,719 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BE65D4E3697E4277ACD95B3D75CC8F23
2016-01-14 15:21:42,719 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:42,721 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784900749_0001_1_01_000000_0
2016-01-14 15:21:42,725 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452784900749_0001_1_00_000000_0_10003, runDuration: 327507, ]
2016-01-14 15:21:42,727 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-01-14 15:21:42,737 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: D50720A172AE4E32A5A403804778ADB5
2016-01-14 15:21:42,737 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: 74BF1F07083B43BA80F36B0CA7880C18
2016-01-14 15:21:42,738 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] 74BF1F07083B43BA80F36B0CA7880C18
2016-01-14 15:21:42,740 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@2d8fcb6; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@73b7122f; comparator=cascading.tuple.hadoop.util.TupleComparator@780b0702; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:21:42,740 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:21:42,741 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:21:42,741 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:21:42,747 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452784900749_0001_1_00_000000_0_10003] to DISK_DIRECT, CompressedSize=204, DecompressedSize=200,EndTime=1452784902747, TimeTaken=10, Rate=0.02 MB/s
2016-01-14 15:21:42,784 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : BE65D4E3697E4277ACD95B3D75CC8F23
2016-01-14 15:21:42,784 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 16515068(66060272)
2016-01-14 15:21:42,787 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 63
2016-01-14 15:21:42,787 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 52848232
2016-01-14 15:21:42,788 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 66060288
2016-01-14 15:21:42,788 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 16515068; length = 4128768
2016-01-14 15:21:42,789 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] D50720A172AE4E32A5A403804778ADB5, for 1 inputs
2016-01-14 15:21:42,790 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:21:42,790 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [BE65D4E3697E4277ACD95B3D75CC8F23] #1 in 54ms
2016-01-14 15:21:42,791 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [BE65D4E3697E4277ACD95B3D75CC8F23] #1, status:false, isInterrupted:false
2016-01-14 15:21:42,791 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:21:42,794 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 204 bytes from disk
2016-01-14 15:21:42,795 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:21:42,799 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:21:42,803 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 175 bytes
2016-01-14 15:21:42,803 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A9A1ADFCDDDF4A0E8658ED305245FDD4, all 1 inputs ready in: 00:00:00.014
2016-01-14 15:21:43,061 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : BE65D4E3697E4277ACD95B3D75CC8F23
2016-01-14 15:21:43,063 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:21:43,064 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@7ef28d8b; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@662ae1c9
2016-01-14 15:21:43,085 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784900749_0001_1_01_000000_0
2016-01-14 15:21:43,086 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A9A1ADFCDDDF4A0E8658ED305245FDD4, mem on close (mb), free: 304, total: 383, max: 672
2016-01-14 15:21:43,088 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: BE65D4E3697E4277ACD95B3D75CC8F23
2016-01-14 15:21:43,090 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:21:43,091 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:21:43,093 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:21:43,096 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:21:43,096 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:21:43,099 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:21:43,099 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:21:43,099 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 184; bufvoid = 66060288
2016-01-14 15:21:43,100 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 16515068(66060272); kvend = 16515040(66060160); length = 29/4128768
2016-01-14 15:21:43,108 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:21:43 Completed running task attempt: attempt_1452784900749_0001_1_01_000000_0
2016-01-14 15:21:43,113 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784900749_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:43,115 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784900749_0001_1_01, eventCount=1
2016-01-14 15:21:43,115 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:43,119 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A9A1ADFCDDDF4A0E8658ED305245FDD4, taskAttemptId=attempt_1452784900749_0001_1_01_000000_0, startTime=1452784902625, finishTime=1452784903115, timeTaken=490, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 49, File System Counters, FILE_BYTES_READ=268, FILE_BYTES_WRITTEN=242, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=256, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=402128896, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=88, MERGE_PHASE_TIME=101, FIRST_EVENT_RECEIVED=34, LAST_EVENT_RECEIVED=34, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784903060, Process_Duration=25, Process_End_Time=1452784903085, Read_Duration=3, Tuples_Read=10, Tuples_Written=8, Write_Duration=1
2016-01-14 15:21:43,119 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 49 [[File System Counters FILE_BYTES_READ=268, FILE_BYTES_WRITTEN=242, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=256, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=402128896, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=88, MERGE_PHASE_TIME=101, FIRST_EVENT_RECEIVED=34, LAST_EVENT_RECEIVED=34][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452784903060, Process_Duration=25, Process_End_Time=1452784903085, Read_Duration=3, Tuples_Read=10, Tuples_Written=8, Write_Duration=1]]
2016-01-14 15:21:43,119 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784900749_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:43,120 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:43,121 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:43,122 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784900749_0001_00_000002
2016-01-14 15:21:43,120 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784900749_0001_1_01_000000_0
2016-01-14 15:21:43,120 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:43,123 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:43,125 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:TASK_FINISHED]: vertexName=A9A1ADFCDDDF4A0E8658ED305245FDD4, taskId=task_1452784900749_0001_1_01_000000, startTime=1452784902625, finishTime=1452784903122, timeTaken=497, status=SUCCEEDED, successfulAttemptID=attempt_1452784900749_0001_1_01_000000_0, diagnostics=, counters=Counters: 49, File System Counters, FILE_BYTES_READ=268, FILE_BYTES_WRITTEN=242, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=256, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=402128896, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=88, MERGE_PHASE_TIME=101, FIRST_EVENT_RECEIVED=34, LAST_EVENT_RECEIVED=34, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784903060, Process_Duration=25, Process_End_Time=1452784903085, Read_Duration=3, Tuples_Read=10, Tuples_Written=8, Write_Duration=1
2016-01-14 15:21:43,125 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784900749_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:43,125 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784900749_0001_1_01 [A9A1ADFCDDDF4A0E8658ED305245FDD4] : 1
2016-01-14 15:21:43,126 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784900749_0001_1_01 [A9A1ADFCDDDF4A0E8658ED305245FDD4]
2016-01-14 15:21:43,127 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:43,133 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:VERTEX_FINISHED]: vertexName=A9A1ADFCDDDF4A0E8658ED305245FDD4, vertexId=vertex_1452784900749_0001_1_01, initRequestedTime=1452784901834, initedTime=1452784901891, startRequestedTime=1452784902045, startedTime=1452784902045, finishTime=1452784903126, timeTaken=1081, status=SUCCEEDED, diagnostics=, counters=Counters: 49, File System Counters, FILE_BYTES_READ=268, FILE_BYTES_WRITTEN=242, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=256, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=402128896, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=88, MERGE_PHASE_TIME=101, FIRST_EVENT_RECEIVED=34, LAST_EVENT_RECEIVED=34, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784903060, Process_Duration=25, Process_End_Time=1452784903085, Read_Duration=3, Tuples_Read=10, Tuples_Written=8, Write_Duration=1, vertexStats=firstTaskStartTime=1452784902625, firstTasksToStart=[ task_1452784900749_0001_1_01_000000 ], lastTaskFinishTime=1452784903115, lastTasksToFinish=[ task_1452784900749_0001_1_01_000000 ], minTaskDuration=490, maxTaskDuration=490, avgTaskDuration=490.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784900749_0001_1_01_000000 ], longestDurationTasks=[ task_1452784900749_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:43,133 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784900749_0001_1_01 [A9A1ADFCDDDF4A0E8658ED305245FDD4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:43,134 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784900749_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:43,134 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:43,134 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784900749_0001_00_000002, stoppedTime=1452784903134, exitStatus=0
2016-01-14 15:21:43,134 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784900749_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:43,135 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452784900749_0001_1_02 [DA881B27594E4E0EBAF60521FB2E6C42] attempt: attempt_1452784900749_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:21:43,135 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: DA881B27594E4E0EBAF60521FB2E6C42
2016-01-14 15:21:43,135 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784900749_0001_1_02 [DA881B27594E4E0EBAF60521FB2E6C42]
2016-01-14 15:21:43,135 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784900749_0001_1_01 [A9A1ADFCDDDF4A0E8658ED305245FDD4] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 15:21:43,136 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452784900749_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 15:21:43,136 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:43,136 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:TASK_STARTED]: vertexName=DA881B27594E4E0EBAF60521FB2E6C42, taskId=task_1452784900749_0001_1_02_000000, scheduledTime=1452784903136, launchTime=1452784903136
2016-01-14 15:21:43,137 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784900749_0001_1_02_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:43,137 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784900749_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:21:43,137 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784900749_0001_1_02 [DA881B27594E4E0EBAF60521FB2E6C42]
2016-01-14 15:21:43,137 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D1D5FAED70EA45E9BB5FD560F8143B83/BED9C9F811E94D3E810B993D1E951BB7] test/(1/1) ...stSimpleChain/simplechain, VertexName: DA881B27594E4E0EBAF60521FB2E6C42, VertexParallelism: 1, TaskAttemptID:attempt_1452784900749_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A9A1ADFCDDDF4A0E8658ED305245FDD4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=4A465710519144119168DF5824F2FB92, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:43,137 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784900749_0001_1_02_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:43,138 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:43,139 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784900749_0001_00_000003
2016-01-14 15:21:43,139 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784900749_0001_00_000003 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:43,140 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784900749_0001_00_000003
2016-01-14 15:21:43,141 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784900749_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:21:43,141 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784900749_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:43,141 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784900749_0001_00_000002, Exiting
2016-01-14 15:21:43,141 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784900749_0001_00_000002
2016-01-14 15:21:43,141 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784900749_0001_00_000002
2016-01-14 15:21:43,141 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 15:21:43,146 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784900749_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:43,146 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784900749_0001_00_000002 completed successfully
2016-01-14 15:21:43,146 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784900749_0001_00_000003 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:43,146 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784900749_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:21:43,148 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:43,148 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784900749_0001_00_000003, launchTime=1452784903148
2016-01-14 15:21:43,149 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784900749_0001_00_000003
2016-01-14 15:21:43,150 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784900749_0001_1_02_000000_0] to container: [container_1452784900749_0001_00_000003]
2016-01-14 15:21:43,150 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784900749_0001_00_000003 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:43,150 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784900749_0001_00_000003 given task: attempt_1452784900749_0001_1_02_000000_0
2016-01-14 15:21:43,150 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784900749_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784900749_0001_1_02_000000_0
2016-01-14 15:21:43,151 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784900749_0001_1_02_000000_0] started. Is using containerId: [container_1452784900749_0001_00_000003] on NM: [127.0.0.1:0]
2016-01-14 15:21:43,151 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DA881B27594E4E0EBAF60521FB2E6C42, taskAttemptId=attempt_1452784900749_0001_1_02_000000_0, startTime=1452784903150, containerId=container_1452784900749_0001_00_000003, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784900749_0001_00_000003/teamcity, completedLogs=
2016-01-14 15:21:43,151 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784900749_0001_1_02_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:43,151 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784900749_0001_1_02_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:43,152 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:43 Starting to run new task attempt: attempt_1452784900749_0001_1_02_000000_0
2016-01-14 15:21:43,152 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784900749_0001_1_02_000000_0
2016-01-14 15:21:43,152 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:43,152 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:43,153 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:43,153 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:43,153 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:43,153 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:43,153 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D1D5FAED70EA45E9BB5FD560F8143B83/BED9C9F811E94D3E810B993D1E951BB7] test/(1/1) ...stSimpleChain/simplechain, VertexName: DA881B27594E4E0EBAF60521FB2E6C42, VertexParallelism: 1, TaskAttemptID:attempt_1452784900749_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A9A1ADFCDDDF4A0E8658ED305245FDD4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=4A465710519144119168DF5824F2FB92, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:43,153 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:43,154 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784900749_0001_1_02_000000_0
2016-01-14 15:21:43,156 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:43,156 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:43,156 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784900749_0001_1_02_000000_0 sent events: (0-1)
2016-01-14 15:21:43,156 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A9A1ADFCDDDF4A0E8658ED305245FDD4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:21:43,156 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:43,156 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=4A465710519144119168DF5824F2FB92, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:21:43,157 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:43,157 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A9A1ADFCDDDF4A0E8658ED305245FDD4
2016-01-14 15:21:43,157 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:43,157 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 4A465710519144119168DF5824F2FB92
2016-01-14 15:21:43,158 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:21:43,158 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 4A465710519144119168DF5824F2FB92
2016-01-14 15:21:43,158 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:21:43,158 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:21:43,163 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: A9A1ADFCDDDF4A0E8658ED305245FDD4
2016-01-14 15:21:43,160 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:43,164 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A9A1ADFCDDDF4A0E8658ED305245FDD4
2016-01-14 15:21:43,197 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:21:43,197 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 4A465710519144119168DF5824F2FB92
2016-01-14 15:21:43,202 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DA881B27594E4E0EBAF60521FB2E6C42, ordinal: 2
2016-01-14 15:21:43,202 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DA881B27594E4E0EBAF60521FB2E6C42, mem on start (mb), free: 296, total: 383, max: 672
2016-01-14 15:21:43,203 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:43,203 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:43,203 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:43,203 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:43,203 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:43,203 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:21:43,204 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:43,204 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:21:43,204 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:21:43,204 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:21:43,204 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 4A465710519144119168DF5824F2FB92, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:21:43,208 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A9A1ADFCDDDF4A0E8658ED305245FDD4, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:21:43,208 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:43,208 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A9A1ADFCDDDF4A0E8658ED305245FDD4
2016-01-14 15:21:43,210 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:21:43,210 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:21:43,211 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: A9A1ADFCDDDF4A0E8658ED305245FDD4 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:21:43,211 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:21:43,212 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:21:43,212 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:21:43,214 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A9A1ADFCDDDF4A0E8658ED305245FDD4
2016-01-14 15:21:43,214 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A9A1ADFCDDDF4A0E8658ED305245FDD4 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:43,214 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:21:43,215 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:43,215 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:43,217 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784900749_0001_1_02_000000_0
2016-01-14 15:21:43,217 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452784900749_0001_1_01_000000_0_10005, runDuration: 463270, ]
2016-01-14 15:21:43,223 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452784900749_0001_1_01_000000_0_10005] to DISK_DIRECT, CompressedSize=210, DecompressedSize=206,EndTime=1452784903223, TimeTaken=2, Rate=0.10 MB/s
2016-01-14 15:21:43,223 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : A9A1ADFCDDDF4A0E8658ED305245FDD4
2016-01-14 15:21:43,223 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-01-14 15:21:43,224 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:21:43,225 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [A9A1ADFCDDDF4A0E8658ED305245FDD4] #2 in 4ms
2016-01-14 15:21:43,226 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: 74BF1F07083B43BA80F36B0CA7880C18
2016-01-14 15:21:43,226 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain"], id: 4A465710519144119168DF5824F2FB92
2016-01-14 15:21:43,227 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain"]
2016-01-14 15:21:43,228 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] 74BF1F07083B43BA80F36B0CA7880C18, for 1 inputs
2016-01-14 15:21:43,229 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [A9A1ADFCDDDF4A0E8658ED305245FDD4] #2, status:false, isInterrupted:false
2016-01-14 15:21:43,229 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:21:43,230 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 210 bytes from disk
2016-01-14 15:21:43,230 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:21:43,230 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:21:43,231 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 181 bytes
2016-01-14 15:21:43,231 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : A9A1ADFCDDDF4A0E8658ED305245FDD4
2016-01-14 15:21:43,231 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DA881B27594E4E0EBAF60521FB2E6C42, all 1 inputs ready in: 00:00:00.003
2016-01-14 15:21:43,231 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:21:43,237 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@13077cd2; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@5093c8c0
2016-01-14 15:21:43,241 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:21:43,241 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:21:43,242 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452784900749_0001_1_02_000000_0
2016-01-14 15:21:43,243 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452784900749_0001_1_02_000000_0 given a go for committing the task output.
2016-01-14 15:21:43,244 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784900749_0001_1_02_000000_0
2016-01-14 15:21:43,244 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DA881B27594E4E0EBAF60521FB2E6C42, mem on close (mb), free: 294, total: 383, max: 672
2016-01-14 15:21:43,245 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: A9A1ADFCDDDF4A0E8658ED305245FDD4
2016-01-14 15:21:43,245 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:21:43,245 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:21:43,246 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:21:43,246 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:21:43,246 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:21:43 Completed running task attempt: attempt_1452784900749_0001_1_02_000000_0
2016-01-14 15:21:43,248 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784900749_0001_1_02_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:43,254 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:43,256 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DA881B27594E4E0EBAF60521FB2E6C42, taskAttemptId=attempt_1452784900749_0001_1_02_000000_0, startTime=1452784903150, finishTime=1452784903253, timeTaken=103, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=242, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=402128896, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=21, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784903231, Process_Duration=13, Process_End_Time=1452784903244, Read_Duration=1, Tuples_Read=8, Tuples_Written=8, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=8
2016-01-14 15:21:43,256 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784900749_0001_1_02_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:43,257 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784900749_0001_1_02_000000_0
2016-01-14 15:21:43,257 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:43,259 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:43,259 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:TASK_FINISHED]: vertexName=DA881B27594E4E0EBAF60521FB2E6C42, taskId=task_1452784900749_0001_1_02_000000, startTime=1452784903150, finishTime=1452784903257, timeTaken=107, status=SUCCEEDED, successfulAttemptID=attempt_1452784900749_0001_1_02_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=242, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=402128896, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=21, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784903231, Process_Duration=13, Process_End_Time=1452784903244, Read_Duration=1, Tuples_Read=8, Tuples_Written=8, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=8
2016-01-14 15:21:43,260 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784900749_0001_1_02_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:43,260 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784900749_0001_1_02 [DA881B27594E4E0EBAF60521FB2E6C42] : 1
2016-01-14 15:21:43,260 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784900749_0001_1_02 [DA881B27594E4E0EBAF60521FB2E6C42]
2016-01-14 15:21:43,262 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:43,264 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=242, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=402128896, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=21, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452784903231, Process_Duration=13, Process_End_Time=1452784903244, Read_Duration=1, Tuples_Read=8, Tuples_Written=8, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=8]]
2016-01-14 15:21:43,263 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:VERTEX_FINISHED]: vertexName=DA881B27594E4E0EBAF60521FB2E6C42, vertexId=vertex_1452784900749_0001_1_02, initRequestedTime=1452784901894, initedTime=1452784901991, startRequestedTime=1452784902062, startedTime=1452784902062, finishTime=1452784903260, timeTaken=1198, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=242, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=402128896, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=21, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784903231, Process_Duration=13, Process_End_Time=1452784903244, Read_Duration=1, Tuples_Read=8, Tuples_Written=8, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=8, vertexStats=firstTaskStartTime=1452784903150, firstTasksToStart=[ task_1452784900749_0001_1_02_000000 ], lastTaskFinishTime=1452784903253, lastTasksToFinish=[ task_1452784900749_0001_1_02_000000 ], minTaskDuration=103, maxTaskDuration=103, avgTaskDuration=103.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784900749_0001_1_02_000000 ], longestDurationTasks=[ task_1452784900749_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:43,266 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784900749_0001_1_02 [DA881B27594E4E0EBAF60521FB2E6C42] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:43,267 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784900749_0001_00_000003 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:43,267 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784900749_0001_00_000003, stoppedTime=1452784903267, exitStatus=0
2016-01-14 15:21:43,267 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:43,269 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784900749_0001_00_000003 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:43,269 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784900749_0001_1_02 [DA881B27594E4E0EBAF60521FB2E6C42] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 15:21:43,269 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452784900749_0001_1
2016-01-14 15:21:43,275 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:43,275 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:43,276 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452784900749_0001_1
2016-01-14 15:21:43,276 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 4A465710519144119168DF5824F2FB92 for vertex: vertex_1452784900749_0001_1_02 [DA881B27594E4E0EBAF60521FB2E6C42]
2016-01-14 15:21:43,276 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784900749_0001_00_000003
2016-01-14 15:21:43,278 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784900749_0001_00_000003 is valid, but no longer registered, and will be killed
2016-01-14 15:21:43,278 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784900749_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:43,278 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784900749_0001_00_000003, Exiting
2016-01-14 15:21:43,278 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784900749_0001_00_000003
2016-01-14 15:21:43,278 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784900749_0001_00_000003
2016-01-14 15:21:43,280 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784900749_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:43,280 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784900749_0001_00_000003 completed successfully
2016-01-14 15:21:43,289 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452784900749_0001_1_01 [A9A1ADFCDDDF4A0E8658ED305245FDD4]
2016-01-14 15:21:43,289 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452784900749_0001_1_00 [BE65D4E3697E4277ACD95B3D75CC8F23]
2016-01-14 15:21:43,296 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452784900749_0001_1, queueSize=0
2016-01-14 15:21:43,297 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:21:43,299 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784900749_0001_1][Event:DAG_FINISHED]: dagId=dag_1452784900749_0001_1, startTime=1452784901789, finishTime=1452784903289, timeTaken=1500, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2716, FILE_BYTES_WRITTEN=1654, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=14, REDUCE_INPUT_RECORDS=18, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=36, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=256, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1099431936, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=26, OUTPUT_BYTES=394, OUTPUT_BYTES_WITH_OVERHEAD=406, OUTPUT_BYTES_PHYSICAL=414, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=414, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=414, SHUFFLE_BYTES_DECOMPRESSED=406, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=414, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=104, MERGE_PHASE_TIME=122, FIRST_EVENT_RECEIVED=42, LAST_EVENT_RECEIVED=42, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4358354708802, Process_Duration=46, Process_End_Time=4358354708848, Read_Duration=6, Tuples_Read=28, Tuples_Written=26, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=8
2016-01-14 15:21:43,300 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452784900749_0001_1 finished with state: SUCCEEDED
2016-01-14 15:21:43,300 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784900749_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:21:43,300 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452784900749_0001_00_000003
2016-01-14 15:21:43,300 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784900749_0001_00_000003 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:21:43 Completed Dag: dag_1452784900749_0001_1
2016-01-14 15:21:43,303 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452784900749_0001_1, dagState=SUCCEEDED
2016-01-14 15:21:43,304 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784900749_0001_1_post
2016-01-14 15:21:43,304 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:43,305 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:21:43,312 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...stSimpleChain/simplechain, applicationId=application_1452784900749_0001
2016-01-14 15:21:43,315 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:21:43,316 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:21:43,316 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:21:43,316 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:21:43,316 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:21:43,317 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:21:43,319 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:21:43,320 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:21:43,320 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:21:43,321 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:21:43,321 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:21:43,322 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:21:43,323 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:21:43,330 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D1D5FAED70EA45E9BB5FD560F8143B83/BED9C9F811E94D3E810B993D1E951BB7/.tez/application_1452784900749_0001
2016-01-14 15:21:43,331 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:02.788
2016-01-14 15:21:43,376 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain
2016-01-14 15:21:43,415 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:21:43,416 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:21:43,480 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.031
2016-01-14 15:21:43,519 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.070
2016-01-14 15:21:43,521 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:21:43,521 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:21:43,521 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:21:43,544 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-01-14 15:21:43,560 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count
2016-01-14 15:21:43,634 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] starting
2016-01-14 15:21:43,635 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 15:21:43,637 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count"]
2016-01-14 15:21:43,639 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] parallel execution of steps is enabled: false
2016-01-14 15:21:43,640 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executing total steps: 1
2016-01-14 15:21:43,640 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] allocating management threads: 1
2016-01-14 15:21:43,641 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] starting step: (1/1) ...splatform/testCount/count
2016-01-14 15:21:43,642 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:21:43,672 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tez session mode enabled: true
2016-01-14 15:21:43,678 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:43,679 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:21:43,680 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:21:43,693 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C410F378EDF54B94A87402070A8F511B/A4ECFC88F37C4A5F909B70C0290864BC/.tez/application_1452784903679_0001 doesn't exist and is created
2016-01-14 15:21:43,713 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:21:43,714 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:21:43,715 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C410F378EDF54B94A87402070A8F511B/A4ECFC88F37C4A5F909B70C0290864BC/.tez/application_1452784903679_0001
2016-01-14 15:21:43,720 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452784903679_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:43,733 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:21:43,734 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:21:43,736 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:21:43,737 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:21:43,737 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C410F378EDF54B94A87402070A8F511B/A4ECFC88F37C4A5F909B70C0290864BC/.tez/application_1452784903679_0001/localmode-log-dir
2016-01-14 15:21:43,737 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C410F378EDF54B94A87402070A8F511B/A4ECFC88F37C4A5F909B70C0290864BC/.tez/application_1452784903679_0001/localmode-log-dir/history.txt.appattempt_1452784903679_0001_000000, maxErrors=10
2016-01-14 15:21:43,737 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:21:43,738 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452784903679_0001_000000, appSubmitTime=1452784903719, launchTime=1452784903719
2016-01-14 15:21:43,744 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:21:43,747 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:21:43,750 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:21:43,752 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:21:43,753 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:44859
2016-01-14 15:21:43,758 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:21:43,761 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:21:43,761 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452784903679_0001_000000, startTime=1452784903760
2016-01-14 15:21:43,761 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:21:43,814 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:21:43,814 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:21:43,814 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...splatform/testCount/count, applicationId=application_1452784903679_0001, dagName=[C410F378EDF54B94A87402070A8F511B/A4ECFC88F37C4A5F909B70C0290864BC] count/(1/1) ...splatform/testCount/count
2016-01-14 15:21:43,835 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [C410F378EDF54B94A87402070A8F511B/A4ECFC88F37C4A5F909B70C0290864BC] count/(1/1) ...splatform/testCount/count
2016-01-14 15:21:43,838 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452784903679_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C410F378EDF54B94A87402070A8F511B/A4ECFC88F37C4A5F909B70C0290864BC/.tez/application_1452784903679_0001/localmode-log-dir/dag_1452784903679_0001_1.dot
2016-01-14 15:21:43,840 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C410F378EDF54B94A87402070A8F511B/A4ECFC88F37C4A5F909B70C0290864BC/.tez/application_1452784903679_0001/localmode-log-dir/dag_1452784903679_0001_1-tez-dag.pb.txt
2016-01-14 15:21:43,870 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784903679_0001_1
2016-01-14 15:21:43,870 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:43,870 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [C410F378EDF54B94A87402070A8F511B/A4ECFC88F37C4A5F909B70C0290864BC] count/(1/1) ...splatform/testCount/count
2016-01-14 15:21:43 Running Dag: dag_1452784903679_0001_1
2016-01-14 15:21:43,886 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:21:43,886 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784903679_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452784903679_0001_1, submitTime=1452784903835
2016-01-14 15:21:43,887 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BE387DBC5C284AAEA70B1E1715F23470
2016-01-14 15:21:43,888 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:21:43,888 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784903679_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452784903679_0001_1, initTime=1452784903886
2016-01-14 15:21:43,888 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:21:43,889 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784903679_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:21:43,890 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:21:43,890 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784903679_0001_1][Event:DAG_STARTED]: dagID=dag_1452784903679_0001_1, startTime=1452784903890
2016-01-14 15:21:43,890 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:21:43,891 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784903679_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:21:43,891 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F48D784DD37E4353B82B48311E35FFAC : {8C88EE8DDA2A42138AC2D6A8FA7913A5={InputName=8C88EE8DDA2A42138AC2D6A8FA7913A5}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:21:43,891 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 8C88EE8DDA2A42138AC2D6A8FA7913A5, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:21:43,891 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452784903679_0001_1_00 [F48D784DD37E4353B82B48311E35FFAC]
2016-01-14 15:21:43,891 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452784903679_0001_1_00 [F48D784DD37E4353B82B48311E35FFAC]
2016-01-14 15:21:43,891 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452784903679_0001_1_00 [F48D784DD37E4353B82B48311E35FFAC]
2016-01-14 15:21:43,891 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452784903679_0001_1_00 [F48D784DD37E4353B82B48311E35FFAC]. Starting root input initializers: 1
2016-01-14 15:21:43,892 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784903679_0001_1_00 [F48D784DD37E4353B82B48311E35FFAC] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:21:43,892 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452784903679_0001_1_01 [BE387DBC5C284AAEA70B1E1715F23470]
2016-01-14 15:21:43,892 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...splatform/testCount/count, applicationId=application_1452784903679_0001, dagName=[C410F378EDF54B94A87402070A8F511B/A4ECFC88F37C4A5F909B70C0290864BC] count/(1/1) ...splatform/testCount/count
2016-01-14 15:21:43,898 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:21:43,899 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452784903679_0001_1_01 [BE387DBC5C284AAEA70B1E1715F23470]
2016-01-14 15:21:43,900 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452784903679_0001_1_01 [BE387DBC5C284AAEA70B1E1715F23470]
2016-01-14 15:21:43,900 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452784903679_0001_1_01 [BE387DBC5C284AAEA70B1E1715F23470]
2016-01-14 15:21:43,901 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=3D2270DED3B64139B46857802BDDB8A6, vertexId=vertex_1452784903679_0001_1_01 [BE387DBC5C284AAEA70B1E1715F23470], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:21:43,901 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=3D2270DED3B64139B46857802BDDB8A6, vertexId=vertex_1452784903679_0001_1_01 [BE387DBC5C284AAEA70B1E1715F23470]
2016-01-14 15:21:43,905 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 8C88EE8DDA2A42138AC2D6A8FA7913A5 on vertex vertex_1452784903679_0001_1_00 [F48D784DD37E4353B82B48311E35FFAC]
2016-01-14 15:21:43,908 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:21:43,910 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 8C88EE8DDA2A42138AC2D6A8FA7913A5 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:21:43,910 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:21:43,911 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:21:43,909 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=3D2270DED3B64139B46857802BDDB8A6, vertexName=BE387DBC5C284AAEA70B1E1715F23470, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:21:43,913 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=3D2270DED3B64139B46857802BDDB8A6, vertexId=vertex_1452784903679_0001_1_01 [BE387DBC5C284AAEA70B1E1715F23470]
2016-01-14 15:21:43,914 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-01-14 15:21:43,915 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:43,915 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784903679_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BE387DBC5C284AAEA70B1E1715F23470, vertexId=vertex_1452784903679_0001_1_01, initRequestedTime=1452784903892, initedTime=1452784903914, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:21:43,915 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784903679_0001_1_01 [BE387DBC5C284AAEA70B1E1715F23470] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:21:43,916 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 8C88EE8DDA2A42138AC2D6A8FA7913A5 on vertex vertex_1452784903679_0001_1_00 [F48D784DD37E4353B82B48311E35FFAC]
2016-01-14 15:21:43,916 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784903679_0001_1_00 [F48D784DD37E4353B82B48311E35FFAC] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:43,917 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {8C88EE8DDA2A42138AC2D6A8FA7913A5=forAllWorkUnits=true, update=[1]}
2016-01-14 15:21:43,917 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452784903679_0001_1_00 [F48D784DD37E4353B82B48311E35FFAC] parallelism set to 1
2016-01-14 15:21:43,917 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784903679_0001_1_00 [F48D784DD37E4353B82B48311E35FFAC] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:43,919 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784903679_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784903679_0001_1_00, eventCount=1
2016-01-14 15:21:43,922 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:43,922 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784903679_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F48D784DD37E4353B82B48311E35FFAC, vertexId=vertex_1452784903679_0001_1_00, initRequestedTime=1452784903891, initedTime=1452784903921, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:21:43,923 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784903679_0001_1_00 [F48D784DD37E4353B82B48311E35FFAC] with distanceFromRoot: 0
2016-01-14 15:21:43,923 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784903679_0001_1_00 [F48D784DD37E4353B82B48311E35FFAC] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:21:43,923 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F48D784DD37E4353B82B48311E35FFAC
2016-01-14 15:21:43,923 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784903679_0001_1_00 [F48D784DD37E4353B82B48311E35FFAC]
2016-01-14 15:21:43,924 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:43,924 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784903679_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784903679_0001_1_00, startRequestedTime=1452784903923, startedTime=1452784903923
2016-01-14 15:21:43,925 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] submitted tez dag to app master: application_1452784903679_0001, with dag id: dag_1452784903679_0001_1
2016-01-14 15:21:43,926 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784903679_0001_1_00 [F48D784DD37E4353B82B48311E35FFAC] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:43,927 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:43,928 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784903679_0001_1][Event:TASK_STARTED]: vertexName=F48D784DD37E4353B82B48311E35FFAC, taskId=task_1452784903679_0001_1_00_000000, scheduledTime=1452784903927, launchTime=1452784903927
2016-01-14 15:21:43,929 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784903679_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:43,929 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452784903679_0001_1_00 for vertex: vertex_1452784903679_0001_1_01 [BE387DBC5C284AAEA70B1E1715F23470] numStartedSources: 1 numSources: 1
2016-01-14 15:21:43,929 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784903679_0001_1_01 [BE387DBC5C284AAEA70B1E1715F23470] with distanceFromRoot: 1
2016-01-14 15:21:43,931 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784903679_0001_1_00 [F48D784DD37E4353B82B48311E35FFAC]
2016-01-14 15:21:43,933 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F48D784DD37E4353B82B48311E35FFAC in vertex: BE387DBC5C284AAEA70B1E1715F23470
2016-01-14 15:21:43,934 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: BE387DBC5C284AAEA70B1E1715F23470 with 1 source tasks and 1 pending tasks
2016-01-14 15:21:43,935 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:43,936 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784903679_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784903679_0001_1_01, startRequestedTime=1452784903933, startedTime=1452784903933
2016-01-14 15:21:43,940 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784903679_0001_1_01 [BE387DBC5C284AAEA70B1E1715F23470] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:43,941 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C410F378EDF54B94A87402070A8F511B/A4ECFC88F37C4A5F909B70C0290864BC] count/(1/1) ...splatform/testCount/count, VertexName: F48D784DD37E4353B82B48311E35FFAC, VertexParallelism: 1, TaskAttemptID:attempt_1452784903679_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8C88EE8DDA2A42138AC2D6A8FA7913A5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BE387DBC5C284AAEA70B1E1715F23470, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:21:43,942 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784903679_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:43,943 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:43,944 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:21:43,945 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:21:43,946 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:21:43,947 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784903679_0001_00_000001
2016-01-14 15:21:43,947 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784903679_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:43,950 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784903679_0001_00_000001
2016-01-14 15:21:43,950 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:21:43,951 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:43,951 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784903679_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784903679_0001_00_000001, launchTime=1452784903951
2016-01-14 15:21:43,952 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784903679_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:43,953 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784903679_0001_00_000001
2016-01-14 15:21:43,954 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784903679_0001_1_00_000000_0] to container: [container_1452784903679_0001_00_000001]
2016-01-14 15:21:43,954 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784903679_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:43,954 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784903679_0001_00_000001 given task: attempt_1452784903679_0001_1_00_000000_0
2016-01-14 15:21:43,955 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784903679_0001_1_00_000000_0] started. Is using containerId: [container_1452784903679_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:21:43,955 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784903679_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F48D784DD37E4353B82B48311E35FFAC, taskAttemptId=attempt_1452784903679_0001_1_00_000000_0, startTime=1452784903955, containerId=container_1452784903679_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784903679_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:21:43,956 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784903679_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:43,956 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:43,956 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784903679_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:43,956 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784903679_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784903679_0001_1_00_000000_0
2016-01-14 15:21:43 Starting to run new task attempt: attempt_1452784903679_0001_1_00_000000_0
2016-01-14 15:21:43,957 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784903679_0001_1_00_000000_0
2016-01-14 15:21:43,957 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:43,958 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:43,958 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:43,959 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:43,961 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:43,962 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:43,962 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C410F378EDF54B94A87402070A8F511B/A4ECFC88F37C4A5F909B70C0290864BC] count/(1/1) ...splatform/testCount/count, VertexName: F48D784DD37E4353B82B48311E35FFAC, VertexParallelism: 1, TaskAttemptID:attempt_1452784903679_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8C88EE8DDA2A42138AC2D6A8FA7913A5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BE387DBC5C284AAEA70B1E1715F23470, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:21:43,963 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:43,963 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784903679_0001_1_00_000000_0
2016-01-14 15:21:43,965 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:43,965 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:43,967 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784903679_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:21:43,969 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=BE387DBC5C284AAEA70B1E1715F23470, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:21:43,968 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:43,967 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=8C88EE8DDA2A42138AC2D6A8FA7913A5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:21:43,972 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:43,972 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 8C88EE8DDA2A42138AC2D6A8FA7913A5
2016-01-14 15:21:43,972 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 8C88EE8DDA2A42138AC2D6A8FA7913A5
2016-01-14 15:21:43,974 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:43,975 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:43,975 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: BE387DBC5C284AAEA70B1E1715F23470
2016-01-14 15:21:43,978 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:21:43,979 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 8C88EE8DDA2A42138AC2D6A8FA7913A5
2016-01-14 15:21:43,980 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 8C88EE8DDA2A42138AC2D6A8FA7913A5
2016-01-14 15:21:43,984 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:21:43,984 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: BE387DBC5C284AAEA70B1E1715F23470
2016-01-14 15:21:43,987 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: BE387DBC5C284AAEA70B1E1715F23470
2016-01-14 15:21:43,989 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F48D784DD37E4353B82B48311E35FFAC, ordinal: 0
2016-01-14 15:21:43,989 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F48D784DD37E4353B82B48311E35FFAC, mem on start (mb), free: 206, total: 383, max: 672
2016-01-14 15:21:43,989 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:43,990 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:43,990 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:43,991 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:43,991 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:43,992 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:21:43,992 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:43,992 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:21:43,993 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:21:43,993 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:21:43,994 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 8C88EE8DDA2A42138AC2D6A8FA7913A5, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:21:43,994 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, BE387DBC5C284AAEA70B1E1715F23470, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:21:43,995 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:43,995 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 8C88EE8DDA2A42138AC2D6A8FA7913A5
2016-01-14 15:21:43,995 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:43,995 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 8C88EE8DDA2A42138AC2D6A8FA7913A5
2016-01-14 15:21:43,996 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 8C88EE8DDA2A42138AC2D6A8FA7913A5 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:43,996 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:21:43,997 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:43,997 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:43,997 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784903679_0001_1_00_000000_0
2016-01-14 15:21:44,006 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:21:44,006 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 15:21:44,010 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: 8C88EE8DDA2A42138AC2D6A8FA7913A5
2016-01-14 15:21:44,011 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: EDE8E330CD2C477D90D45E3C15DCD888
2016-01-14 15:21:44,012 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] EDE8E330CD2C477D90D45E3C15DCD888
2016-01-14 15:21:44,011 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6b23be1a
2016-01-14 15:21:44,014 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@3e4c2521; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@7919b97a; comparator=cascading.tuple.hadoop.util.TupleComparator@7e036b3; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:21:44,016 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:21:44,016 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:21:44,017 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:21:44,015 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6b23be1a
2016-01-14 15:21:44,023 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:21:44,023 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:21:44,037 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:21:44,038 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:21:44,038 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:21:44,041 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:21:44,042 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:21:44,044 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 15:21:44,044 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:44,044 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F48D784DD37E4353B82B48311E35FFAC, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:21:44,048 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784903679_0001_1_00_000000_0
2016-01-14 15:21:44,048 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F48D784DD37E4353B82B48311E35FFAC, mem on close (mb), free: 104, total: 383, max: 672
2016-01-14 15:21:44,048 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:21:44,049 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:21:44,049 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2016-01-14 15:21:44,049 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 15:21:44,056 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:21:44 Completed running task attempt: attempt_1452784903679_0001_1_00_000000_0
2016-01-14 15:21:44,068 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784903679_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:44,069 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784903679_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784903679_0001_1_00, eventCount=1
2016-01-14 15:21:44,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=402128896, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452784904045, Process_Duration=3, Process_End_Time=1452784904048, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 15:21:44,071 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784903679_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F48D784DD37E4353B82B48311E35FFAC, taskAttemptId=attempt_1452784903679_0001_1_00_000000_0, startTime=1452784903955, finishTime=1452784904069, timeTaken=114, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=402128896, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784904045, Process_Duration=3, Process_End_Time=1452784904048, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 15:21:44,071 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784903679_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:44,072 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:44,069 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:44,072 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784903679_0001_1_00_000000_0
2016-01-14 15:21:44,074 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784903679_0001_1][Event:TASK_FINISHED]: vertexName=F48D784DD37E4353B82B48311E35FFAC, taskId=task_1452784903679_0001_1_00_000000, startTime=1452784903955, finishTime=1452784904073, timeTaken=118, status=SUCCEEDED, successfulAttemptID=attempt_1452784903679_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=402128896, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784904045, Process_Duration=3, Process_End_Time=1452784904048, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 15:21:44,075 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784903679_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:44,075 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784903679_0001_1_00 [F48D784DD37E4353B82B48311E35FFAC] : 1
2016-01-14 15:21:44,075 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784903679_0001_1_00 [F48D784DD37E4353B82B48311E35FFAC]
2016-01-14 15:21:44,071 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:44,077 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:44,077 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:44,077 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784903679_0001_1][Event:VERTEX_FINISHED]: vertexName=F48D784DD37E4353B82B48311E35FFAC, vertexId=vertex_1452784903679_0001_1_00, initRequestedTime=1452784903891, initedTime=1452784903921, startRequestedTime=1452784903923, startedTime=1452784903923, finishTime=1452784904075, timeTaken=152, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=402128896, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784904045, Process_Duration=3, Process_End_Time=1452784904048, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452784903955, firstTasksToStart=[ task_1452784903679_0001_1_00_000000 ], lastTaskFinishTime=1452784904069, lastTasksToFinish=[ task_1452784903679_0001_1_00_000000 ], minTaskDuration=114, maxTaskDuration=114, avgTaskDuration=114.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784903679_0001_1_00_000000 ], longestDurationTasks=[ task_1452784903679_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:44,078 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784903679_0001_1_00 [F48D784DD37E4353B82B48311E35FFAC] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:44,078 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784903679_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:44,078 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:44,080 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784903679_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784903679_0001_00_000001, stoppedTime=1452784904080, exitStatus=0
2016-01-14 15:21:44,080 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784903679_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:44,081 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452784903679_0001_1_01 [BE387DBC5C284AAEA70B1E1715F23470] attempt: attempt_1452784903679_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:21:44,081 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: BE387DBC5C284AAEA70B1E1715F23470
2016-01-14 15:21:44,081 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784903679_0001_1_01 [BE387DBC5C284AAEA70B1E1715F23470]
2016-01-14 15:21:44,081 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:44,082 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784903679_0001_1_00 [F48D784DD37E4353B82B48311E35FFAC] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:21:44,082 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452784903679_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:21:44,082 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:44,083 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784903679_0001_1][Event:TASK_STARTED]: vertexName=BE387DBC5C284AAEA70B1E1715F23470, taskId=task_1452784903679_0001_1_01_000000, scheduledTime=1452784904082, launchTime=1452784904082
2016-01-14 15:21:44,083 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784903679_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:44,083 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784903679_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:21:44,083 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784903679_0001_1_01 [BE387DBC5C284AAEA70B1E1715F23470]
2016-01-14 15:21:44,083 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C410F378EDF54B94A87402070A8F511B/A4ECFC88F37C4A5F909B70C0290864BC] count/(1/1) ...splatform/testCount/count, VertexName: BE387DBC5C284AAEA70B1E1715F23470, VertexParallelism: 1, TaskAttemptID:attempt_1452784903679_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F48D784DD37E4353B82B48311E35FFAC, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=3D2270DED3B64139B46857802BDDB8A6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:44,084 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784903679_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:44,084 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:44,086 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784903679_0001_00_000001
2016-01-14 15:21:44,087 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784903679_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:21:44,087 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784903679_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:44,087 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784903679_0001_00_000001, Exiting
2016-01-14 15:21:44,087 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784903679_0001_00_000001
2016-01-14 15:21:44,087 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784903679_0001_00_000001
2016-01-14 15:21:44,095 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784903679_0001_00_000002
2016-01-14 15:21:44,095 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784903679_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:44,096 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784903679_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:44,096 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784903679_0001_00_000001 completed successfully
2016-01-14 15:21:44,096 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784903679_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:21:44,098 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784903679_0001_00_000002
2016-01-14 15:21:44,099 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 15:21:44,102 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784903679_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:44,103 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:44,104 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784903679_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784903679_0001_00_000002, launchTime=1452784904102
2016-01-14 15:21:44,105 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784903679_0001_00_000002
2016-01-14 15:21:44,106 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784903679_0001_1_01_000000_0] to container: [container_1452784903679_0001_00_000002]
2016-01-14 15:21:44,107 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784903679_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:44,108 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784903679_0001_1_01_000000_0] started. Is using containerId: [container_1452784903679_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:21:44,108 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784903679_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BE387DBC5C284AAEA70B1E1715F23470, taskAttemptId=attempt_1452784903679_0001_1_01_000000_0, startTime=1452784904108, containerId=container_1452784903679_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784903679_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:21:44,108 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:44,108 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784903679_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:44,109 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784903679_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:44,109 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784903679_0001_00_000002 given task: attempt_1452784903679_0001_1_01_000000_0
2016-01-14 15:21:44,110 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784903679_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784903679_0001_1_01_000000_0
2016-01-14 15:21:44 Starting to run new task attempt: attempt_1452784903679_0001_1_01_000000_0
2016-01-14 15:21:44,110 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784903679_0001_1_01_000000_0
2016-01-14 15:21:44,111 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:44,111 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:44,111 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:44,111 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:44,111 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:44,111 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:44,111 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C410F378EDF54B94A87402070A8F511B/A4ECFC88F37C4A5F909B70C0290864BC] count/(1/1) ...splatform/testCount/count, VertexName: BE387DBC5C284AAEA70B1E1715F23470, VertexParallelism: 1, TaskAttemptID:attempt_1452784903679_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F48D784DD37E4353B82B48311E35FFAC, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=3D2270DED3B64139B46857802BDDB8A6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:44,112 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:44,112 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784903679_0001_1_01_000000_0
2016-01-14 15:21:44,113 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:44,113 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:44,117 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:44,117 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=3D2270DED3B64139B46857802BDDB8A6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:21:44,117 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784903679_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:21:44,118 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:44,119 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F48D784DD37E4353B82B48311E35FFAC, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:21:44,119 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 3D2270DED3B64139B46857802BDDB8A6
2016-01-14 15:21:44,119 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:21:44,120 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 3D2270DED3B64139B46857802BDDB8A6
2016-01-14 15:21:44,119 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:44,121 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F48D784DD37E4353B82B48311E35FFAC
2016-01-14 15:21:44,121 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:44,131 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:21:44,131 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:21:44,138 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: F48D784DD37E4353B82B48311E35FFAC
2016-01-14 15:21:44,140 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F48D784DD37E4353B82B48311E35FFAC
2016-01-14 15:21:44,142 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:21:44,142 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 3D2270DED3B64139B46857802BDDB8A6
2016-01-14 15:21:44,145 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BE387DBC5C284AAEA70B1E1715F23470, ordinal: 1
2016-01-14 15:21:44,146 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BE387DBC5C284AAEA70B1E1715F23470, mem on start (mb), free: 97, total: 383, max: 672
2016-01-14 15:21:44,146 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:44,147 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:44,147 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:44,148 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:44,148 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:44,149 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:21:44,150 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:44,150 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:21:44,151 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:21:44,152 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:21:44,153 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 3D2270DED3B64139B46857802BDDB8A6, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:21:44,153 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F48D784DD37E4353B82B48311E35FFAC, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:21:44,153 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:44,154 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F48D784DD37E4353B82B48311E35FFAC
2016-01-14 15:21:44,155 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:21:44,156 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F48D784DD37E4353B82B48311E35FFAC being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:44,156 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:21:44,157 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:21:44,158 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:44,160 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F48D784DD37E4353B82B48311E35FFAC with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:21:44,160 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:21:44,160 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:21:44,161 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:21:44,162 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F48D784DD37E4353B82B48311E35FFAC
2016-01-14 15:21:44,164 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:44,164 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784903679_0001_1_01_000000_0
2016-01-14 15:21:44,168 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-01-14 15:21:44,169 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: EDE8E330CD2C477D90D45E3C15DCD888
2016-01-14 15:21:44,170 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count"], id: 3D2270DED3B64139B46857802BDDB8A6
2016-01-14 15:21:44,167 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452784903679_0001_1_00_000000_0_10012, runDuration: 84428, ]
2016-01-14 15:21:44,170 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count"]
2016-01-14 15:21:44,172 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] EDE8E330CD2C477D90D45E3C15DCD888, for 1 inputs
2016-01-14 15:21:44,173 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452784903679_0001_1_00_000000_0_10012] to DISK_DIRECT, CompressedSize=463, DecompressedSize=459,EndTime=1452784904173, TimeTaken=2, Rate=0.22 MB/s
2016-01-14 15:21:44,174 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F48D784DD37E4353B82B48311E35FFAC
2016-01-14 15:21:44,174 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:21:44,174 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [F48D784DD37E4353B82B48311E35FFAC] #3 in 3ms
2016-01-14 15:21:44,174 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F48D784DD37E4353B82B48311E35FFAC] #3, status:false, isInterrupted:false
2016-01-14 15:21:44,175 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:21:44,175 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 463 bytes from disk
2016-01-14 15:21:44,175 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:21:44,176 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:21:44,176 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-01-14 15:21:44,176 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F48D784DD37E4353B82B48311E35FFAC
2016-01-14 15:21:44,176 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:21:44,177 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BE387DBC5C284AAEA70B1E1715F23470, all 1 inputs ready in: 00:00:00.003
2016-01-14 15:21:44,178 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@eff7304; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@24d8134d
2016-01-14 15:21:44,183 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:21:44,183 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:21:44,184 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452784903679_0001_1_01_000000_0
2016-01-14 15:21:44,185 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452784903679_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:21:44,186 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784903679_0001_1_01_000000_0
2016-01-14 15:21:44,186 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BE387DBC5C284AAEA70B1E1715F23470, mem on close (mb), free: 95, total: 383, max: 672
2016-01-14 15:21:44,186 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F48D784DD37E4353B82B48311E35FFAC
2016-01-14 15:21:44,187 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:21:44,187 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:21:44,188 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:21:44,188 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:21:44,188 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:21:44 Completed running task attempt: attempt_1452784903679_0001_1_01_000000_0
2016-01-14 15:21:44,192 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784903679_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:44,196 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=402128896, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=18, FIRST_EVENT_RECEIVED=13, LAST_EVENT_RECEIVED=13][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452784904177, Process_Duration=9, Process_End_Time=1452784904186, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=17]]
2016-01-14 15:21:44,196 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:44,197 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:44,199 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:44,197 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784903679_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BE387DBC5C284AAEA70B1E1715F23470, taskAttemptId=attempt_1452784903679_0001_1_01_000000_0, startTime=1452784904108, finishTime=1452784904196, timeTaken=88, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=402128896, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=18, FIRST_EVENT_RECEIVED=13, LAST_EVENT_RECEIVED=13, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784904177, Process_Duration=9, Process_End_Time=1452784904186, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-01-14 15:21:44,199 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784903679_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:44,199 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:44,199 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784903679_0001_00_000002
2016-01-14 15:21:44,199 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784903679_0001_1_01_000000_0
2016-01-14 15:21:44,201 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784903679_0001_1][Event:TASK_FINISHED]: vertexName=BE387DBC5C284AAEA70B1E1715F23470, taskId=task_1452784903679_0001_1_01_000000, startTime=1452784904108, finishTime=1452784904200, timeTaken=92, status=SUCCEEDED, successfulAttemptID=attempt_1452784903679_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=402128896, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=18, FIRST_EVENT_RECEIVED=13, LAST_EVENT_RECEIVED=13, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784904177, Process_Duration=9, Process_End_Time=1452784904186, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-01-14 15:21:44,201 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784903679_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:44,201 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:44,201 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784903679_0001_1_01 [BE387DBC5C284AAEA70B1E1715F23470] : 1
2016-01-14 15:21:44,205 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784903679_0001_1_01 [BE387DBC5C284AAEA70B1E1715F23470]
2016-01-14 15:21:44,206 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:44,207 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784903679_0001_1][Event:VERTEX_FINISHED]: vertexName=BE387DBC5C284AAEA70B1E1715F23470, vertexId=vertex_1452784903679_0001_1_01, initRequestedTime=1452784903892, initedTime=1452784903914, startRequestedTime=1452784903933, startedTime=1452784903933, finishTime=1452784904205, timeTaken=272, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=402128896, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=18, FIRST_EVENT_RECEIVED=13, LAST_EVENT_RECEIVED=13, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784904177, Process_Duration=9, Process_End_Time=1452784904186, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1452784904108, firstTasksToStart=[ task_1452784903679_0001_1_01_000000 ], lastTaskFinishTime=1452784904196, lastTasksToFinish=[ task_1452784903679_0001_1_01_000000 ], minTaskDuration=88, maxTaskDuration=88, avgTaskDuration=88.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784903679_0001_1_01_000000 ], longestDurationTasks=[ task_1452784903679_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:44,207 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784903679_0001_1_01 [BE387DBC5C284AAEA70B1E1715F23470] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:44,207 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784903679_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:44,208 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784903679_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784903679_0001_00_000002, stoppedTime=1452784904207, exitStatus=0
2016-01-14 15:21:44,208 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784903679_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:44,208 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:44,212 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784903679_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:21:44,213 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784903679_0001_00_000002: 13 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:44,209 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784903679_0001_1_01 [BE387DBC5C284AAEA70B1E1715F23470] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:21:44,214 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452784903679_0001_1
2016-01-14 15:21:44,214 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784903679_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452784903679_0001_1
2016-01-14 15:21:44,214 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784903679_0001_00_000002, Exiting
2016-01-14 15:21:44,214 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784903679_0001_00_000002
2016-01-14 15:21:44,214 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784903679_0001_00_000002
2016-01-14 15:21:44,214 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452784903679_0001_1_00 [F48D784DD37E4353B82B48311E35FFAC]
2016-01-14 15:21:44,215 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 3D2270DED3B64139B46857802BDDB8A6 for vertex: vertex_1452784903679_0001_1_01 [BE387DBC5C284AAEA70B1E1715F23470]
2016-01-14 15:21:44,216 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784903679_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:44,218 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784903679_0001_00_000002 completed successfully
2016-01-14 15:21:44,231 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452784903679_0001_1, queueSize=0
2016-01-14 15:21:44,231 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:21:44,232 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784903679_0001_1][Event:DAG_FINISHED]: dagId=dag_1452784903679_0001_1, startTime=1452784903890, finishTime=1452784904229, timeTaken=339, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=940, FILE_BYTES_WRITTEN=777, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=804257792, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=18, FIRST_EVENT_RECEIVED=13, LAST_EVENT_RECEIVED=13, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905569808222, Process_Duration=12, Process_End_Time=2905569808234, Read_Duration=2, Tuples_Read=40, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-01-14 15:21:44,232 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452784903679_0001_1 finished with state: SUCCEEDED
2016-01-14 15:21:44,232 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784903679_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:21:44,233 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784903679_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:21:44,233 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452784903679_0001_00_000002
2016-01-14 15:21:44 Completed Dag: dag_1452784903679_0001_1
2016-01-14 15:21:44,235 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452784903679_0001_1, dagState=SUCCEEDED
2016-01-14 15:21:44,235 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784903679_0001_1_post
2016-01-14 15:21:44,235 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:44,235 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:21:44,245 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...splatform/testCount/count, applicationId=application_1452784903679_0001
2016-01-14 15:21:44,248 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:21:44,248 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:21:44,248 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:21:44,248 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:21:44,248 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:21:44,250 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:21:44,251 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:21:44,251 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:21:44,251 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:21:44,251 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:21:44,252 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:21:44,252 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:21:44,253 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:21:44,257 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C410F378EDF54B94A87402070A8F511B/A4ECFC88F37C4A5F909B70C0290864BC/.tez/application_1452784903679_0001
2016-01-14 15:21:44,258 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] completed in: 00:00.624
2016-01-14 15:21:44,422 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount/count
2016-01-14 15:21:44,448 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:21:44,448 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:21:44,500 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-01-14 15:21:44,507 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.027
2016-01-14 15:21:44,507 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 15:21:44,508 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 15:21:44,508 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:21:44,527 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-01-14 15:21:44,527 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap
2016-01-14 15:21:44,541 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 15:21:44,541 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 15:21:44,541 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap"]
2016-01-14 15:21:44,542 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 15:21:44,542 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 15:21:44,542 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 15:21:44,542 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...pesplatform/testSwap/swap
2016-01-14 15:21:44,543 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:21:44,557 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 15:21:44,562 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:44,563 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:21:44,563 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:21:44,573 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A1B8C261B8DB4B1F9D9AAAD4B39AC09D/F31FEFAA28AB485F81600DD8960507F0/.tez/application_1452784904563_0001 doesn't exist and is created
2016-01-14 15:21:44,595 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:21:44,595 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:21:44,596 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A1B8C261B8DB4B1F9D9AAAD4B39AC09D/F31FEFAA28AB485F81600DD8960507F0/.tez/application_1452784904563_0001
2016-01-14 15:21:44,600 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452784904563_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:44,612 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:21:44,613 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:21:44,615 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:21:44,615 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:21:44,615 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A1B8C261B8DB4B1F9D9AAAD4B39AC09D/F31FEFAA28AB485F81600DD8960507F0/.tez/application_1452784904563_0001/localmode-log-dir
2016-01-14 15:21:44,616 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A1B8C261B8DB4B1F9D9AAAD4B39AC09D/F31FEFAA28AB485F81600DD8960507F0/.tez/application_1452784904563_0001/localmode-log-dir/history.txt.appattempt_1452784904563_0001_000000, maxErrors=10
2016-01-14 15:21:44,616 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:21:44,616 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452784904563_0001_000000, appSubmitTime=1452784904599, launchTime=1452784904599
2016-01-14 15:21:44,622 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:21:44,624 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:21:44,627 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:21:44,627 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:21:44,633 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:37314
2016-01-14 15:21:44,638 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:21:44,640 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452784904563_0001_000000, startTime=1452784904640
2016-01-14 15:21:44,640 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:21:44,640 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:21:44,695 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:21:44,696 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:21:44,696 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...pesplatform/testSwap/swap, applicationId=application_1452784904563_0001, dagName=[A1B8C261B8DB4B1F9D9AAAD4B39AC09D/F31FEFAA28AB485F81600DD8960507F0] test/(1/1) ...pesplatform/testSwap/swap
2016-01-14 15:21:44,714 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [A1B8C261B8DB4B1F9D9AAAD4B39AC09D/F31FEFAA28AB485F81600DD8960507F0] test/(1/1) ...pesplatform/testSwap/swap
2016-01-14 15:21:44,716 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452784904563_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A1B8C261B8DB4B1F9D9AAAD4B39AC09D/F31FEFAA28AB485F81600DD8960507F0/.tez/application_1452784904563_0001/localmode-log-dir/dag_1452784904563_0001_1.dot
2016-01-14 15:21:44,717 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A1B8C261B8DB4B1F9D9AAAD4B39AC09D/F31FEFAA28AB485F81600DD8960507F0/.tez/application_1452784904563_0001/localmode-log-dir/dag_1452784904563_0001_1-tez-dag.pb.txt
2016-01-14 15:21:44,741 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784904563_0001_1
2016-01-14 15:21:44,741 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:44,742 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [A1B8C261B8DB4B1F9D9AAAD4B39AC09D/F31FEFAA28AB485F81600DD8960507F0] test/(1/1) ...pesplatform/testSwap/swap
2016-01-14 15:21:44 Running Dag: dag_1452784904563_0001_1
2016-01-14 15:21:44,755 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784904563_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452784904563_0001_1, submitTime=1452784904714
2016-01-14 15:21:44,755 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:21:44,755 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A4C743B7D843406AB05846ACCAAD9D35
2016-01-14 15:21:44,755 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:21:44,756 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784904563_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452784904563_0001_1, initTime=1452784904755
2016-01-14 15:21:44,757 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784904563_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:21:44,756 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:21:44,760 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784904563_0001_1][Event:DAG_STARTED]: dagID=dag_1452784904563_0001_1, startTime=1452784904760
2016-01-14 15:21:44,760 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:21:44,761 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...pesplatform/testSwap/swap, applicationId=application_1452784904563_0001, dagName=[A1B8C261B8DB4B1F9D9AAAD4B39AC09D/F31FEFAA28AB485F81600DD8960507F0] test/(1/1) ...pesplatform/testSwap/swap
2016-01-14 15:21:44,761 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784904563_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:21:44,760 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:21:44,761 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A4C743B7D843406AB05846ACCAAD9D35 : {A9B1BFC441924689BE72917371F2D42D={InputName=A9B1BFC441924689BE72917371F2D42D}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:21:44,761 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: A9B1BFC441924689BE72917371F2D42D, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:21:44,761 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452784904563_0001_1_00 [A4C743B7D843406AB05846ACCAAD9D35]
2016-01-14 15:21:44,761 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452784904563_0001_1_00 [A4C743B7D843406AB05846ACCAAD9D35]
2016-01-14 15:21:44,761 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452784904563_0001_1_00 [A4C743B7D843406AB05846ACCAAD9D35]
2016-01-14 15:21:44,762 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452784904563_0001_1_00 [A4C743B7D843406AB05846ACCAAD9D35]. Starting root input initializers: 1
2016-01-14 15:21:44,762 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784904563_0001_1_00 [A4C743B7D843406AB05846ACCAAD9D35] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:21:44,763 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: A9B1BFC441924689BE72917371F2D42D on vertex vertex_1452784904563_0001_1_00 [A4C743B7D843406AB05846ACCAAD9D35]
2016-01-14 15:21:44,765 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:21:44,766 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input A9B1BFC441924689BE72917371F2D42D asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:21:44,767 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:21:44,767 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:21:44,769 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-01-14 15:21:44,770 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: A9B1BFC441924689BE72917371F2D42D on vertex vertex_1452784904563_0001_1_00 [A4C743B7D843406AB05846ACCAAD9D35]
2016-01-14 15:21:44,770 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784904563_0001_1_00 [A4C743B7D843406AB05846ACCAAD9D35] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:44,771 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {A9B1BFC441924689BE72917371F2D42D=forAllWorkUnits=true, update=[1]}
2016-01-14 15:21:44,771 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452784904563_0001_1_00 [A4C743B7D843406AB05846ACCAAD9D35] parallelism set to 1
2016-01-14 15:21:44,771 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784904563_0001_1_00 [A4C743B7D843406AB05846ACCAAD9D35] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:44,771 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784904563_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784904563_0001_1_00, eventCount=1
2016-01-14 15:21:44,772 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452784904563_0001_1_00 [A4C743B7D843406AB05846ACCAAD9D35]
2016-01-14 15:21:44,772 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=DAFB38CED3E84E29B4AB94E1F67EFD6E, vertexId=vertex_1452784904563_0001_1_00 [A4C743B7D843406AB05846ACCAAD9D35], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:21:44,772 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=DAFB38CED3E84E29B4AB94E1F67EFD6E, vertexId=vertex_1452784904563_0001_1_00 [A4C743B7D843406AB05846ACCAAD9D35]
2016-01-14 15:21:44,774 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=DAFB38CED3E84E29B4AB94E1F67EFD6E, vertexName=A4C743B7D843406AB05846ACCAAD9D35, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:21:44,775 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=DAFB38CED3E84E29B4AB94E1F67EFD6E, vertexId=vertex_1452784904563_0001_1_00 [A4C743B7D843406AB05846ACCAAD9D35]
2016-01-14 15:21:44,777 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784904563_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A4C743B7D843406AB05846ACCAAD9D35, vertexId=vertex_1452784904563_0001_1_00, initRequestedTime=1452784904761, initedTime=1452784904777, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:21:44,777 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784904563_0001_1_00 [A4C743B7D843406AB05846ACCAAD9D35] with distanceFromRoot: 0
2016-01-14 15:21:44,777 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:44,778 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784904563_0001_1_00 [A4C743B7D843406AB05846ACCAAD9D35] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:21:44,778 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A4C743B7D843406AB05846ACCAAD9D35
2016-01-14 15:21:44,779 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784904563_0001_1_00 [A4C743B7D843406AB05846ACCAAD9D35]
2016-01-14 15:21:44,779 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:44,779 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784904563_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784904563_0001_1_00, startRequestedTime=1452784904778, startedTime=1452784904778
2016-01-14 15:21:44,780 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784904563_0001_1_00 [A4C743B7D843406AB05846ACCAAD9D35] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:44,783 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:44,782 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452784904563_0001, with dag id: dag_1452784904563_0001_1
2016-01-14 15:21:44,783 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784904563_0001_1][Event:TASK_STARTED]: vertexName=A4C743B7D843406AB05846ACCAAD9D35, taskId=task_1452784904563_0001_1_00_000000, scheduledTime=1452784904783, launchTime=1452784904783
2016-01-14 15:21:44,784 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784904563_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:44,784 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784904563_0001_1_00 [A4C743B7D843406AB05846ACCAAD9D35]
2016-01-14 15:21:44,785 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A1B8C261B8DB4B1F9D9AAAD4B39AC09D/F31FEFAA28AB485F81600DD8960507F0] test/(1/1) ...pesplatform/testSwap/swap, VertexName: A4C743B7D843406AB05846ACCAAD9D35, VertexParallelism: 1, TaskAttemptID:attempt_1452784904563_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A9B1BFC441924689BE72917371F2D42D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DAFB38CED3E84E29B4AB94E1F67EFD6E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:44,785 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784904563_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:44,785 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:44,787 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:21:44,790 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:21:44,791 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:21:44,794 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784904563_0001_00_000001
2016-01-14 15:21:44,794 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784904563_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:44,795 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784904563_0001_00_000001
2016-01-14 15:21:44,796 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:21:44,797 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784904563_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:44,797 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:44,797 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784904563_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784904563_0001_00_000001, launchTime=1452784904797
2016-01-14 15:21:44,798 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784904563_0001_00_000001
2016-01-14 15:21:44,799 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784904563_0001_1_00_000000_0] to container: [container_1452784904563_0001_00_000001]
2016-01-14 15:21:44,799 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784904563_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:44,800 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784904563_0001_00_000001 given task: attempt_1452784904563_0001_1_00_000000_0
2016-01-14 15:21:44,800 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784904563_0001_1_00_000000_0] started. Is using containerId: [container_1452784904563_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:21:44,801 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784904563_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A4C743B7D843406AB05846ACCAAD9D35, taskAttemptId=attempt_1452784904563_0001_1_00_000000_0, startTime=1452784904800, containerId=container_1452784904563_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784904563_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:21:44,801 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784904563_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:44,801 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:44,801 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784904563_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784904563_0001_1_00_000000_0
2016-01-14 15:21:44,801 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784904563_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:44 Starting to run new task attempt: attempt_1452784904563_0001_1_00_000000_0
2016-01-14 15:21:44,802 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784904563_0001_1_00_000000_0
2016-01-14 15:21:44,802 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:44,802 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:44,803 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:44,803 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:44,803 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:44,803 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:44,803 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A1B8C261B8DB4B1F9D9AAAD4B39AC09D/F31FEFAA28AB485F81600DD8960507F0] test/(1/1) ...pesplatform/testSwap/swap, VertexName: A4C743B7D843406AB05846ACCAAD9D35, VertexParallelism: 1, TaskAttemptID:attempt_1452784904563_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A9B1BFC441924689BE72917371F2D42D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DAFB38CED3E84E29B4AB94E1F67EFD6E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:44,803 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:44,804 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784904563_0001_1_00_000000_0
2016-01-14 15:21:44,807 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:44,807 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:44,808 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:44,811 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:44,812 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DAFB38CED3E84E29B4AB94E1F67EFD6E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:21:44,813 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:44,814 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DAFB38CED3E84E29B4AB94E1F67EFD6E
2016-01-14 15:21:44,814 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:21:44,815 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: DAFB38CED3E84E29B4AB94E1F67EFD6E
2016-01-14 15:21:44,812 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784904563_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:21:44,812 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A9B1BFC441924689BE72917371F2D42D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:21:44,820 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:44,821 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A9B1BFC441924689BE72917371F2D42D
2016-01-14 15:21:44,821 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: A9B1BFC441924689BE72917371F2D42D
2016-01-14 15:21:44,819 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A4C743B7D843406AB05846ACCAAD9D35, ordinal: 0
2016-01-14 15:21:44,825 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A4C743B7D843406AB05846ACCAAD9D35, mem on start (mb), free: 283, total: 362, max: 672
2016-01-14 15:21:44,825 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:44,825 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:44,828 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:21:44,828 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: A9B1BFC441924689BE72917371F2D42D
2016-01-14 15:21:44,829 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A9B1BFC441924689BE72917371F2D42D
2016-01-14 15:21:44,829 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:44,834 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:21:44,834 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DAFB38CED3E84E29B4AB94E1F67EFD6E
2016-01-14 15:21:44,834 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:44,835 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:44,835 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:21:44,835 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:44,835 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:21:44,835 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:44,835 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:21:44,836 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-01-14 15:21:44,836 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DAFB38CED3E84E29B4AB94E1F67EFD6E, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:21:44,836 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A9B1BFC441924689BE72917371F2D42D, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:21:44,837 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:44,837 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A9B1BFC441924689BE72917371F2D42D being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:44,837 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A9B1BFC441924689BE72917371F2D42D
2016-01-14 15:21:44,837 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:44,838 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A9B1BFC441924689BE72917371F2D42D
2016-01-14 15:21:44,837 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:21:44,838 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:44,838 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:44,840 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784904563_0001_1_00_000000_0
2016-01-14 15:21:44,840 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:21:44,841 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6914225f
2016-01-14 15:21:44,843 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6914225f
2016-01-14 15:21:44,845 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:21:44,845 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:21:44,845 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 15:21:44,847 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: A9B1BFC441924689BE72917371F2D42D
2016-01-14 15:21:44,847 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap"], id: DAFB38CED3E84E29B4AB94E1F67EFD6E
2016-01-14 15:21:44,847 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap"]
2016-01-14 15:21:44,848 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 15:21:44,849 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:44,849 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A4C743B7D843406AB05846ACCAAD9D35, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:21:44,853 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:21:44,853 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:21:44,854 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452784904563_0001_1_00_000000_0
2016-01-14 15:21:44,854 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452784904563_0001_1_00_000000_0 given a go for committing the task output.
2016-01-14 15:21:44,855 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784904563_0001_1_00_000000_0
2016-01-14 15:21:44,855 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A4C743B7D843406AB05846ACCAAD9D35, mem on close (mb), free: 281, total: 362, max: 672
2016-01-14 15:21:44 Completed running task attempt: attempt_1452784904563_0001_1_00_000000_0
2016-01-14 15:21:44,856 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784904563_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:44,857 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10][cascading.flow.SliceCounters Process_Begin_Time=1452784904849, Process_Duration=6, Process_End_Time=1452784904855, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10, Tuples_Written=10]]
2016-01-14 15:21:44,857 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:44,857 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:44,858 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784904563_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A4C743B7D843406AB05846ACCAAD9D35, taskAttemptId=attempt_1452784904563_0001_1_00_000000_0, startTime=1452784904800, finishTime=1452784904857, timeTaken=57, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452784904849, Process_Duration=6, Process_End_Time=1452784904855, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-01-14 15:21:44,858 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784904563_0001_00_000001
2016-01-14 15:21:44,858 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784904563_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:44,858 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:44,859 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784904563_0001_1_00_000000_0
2016-01-14 15:21:44,859 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:44,860 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:44,860 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784904563_0001_1][Event:TASK_FINISHED]: vertexName=A4C743B7D843406AB05846ACCAAD9D35, taskId=task_1452784904563_0001_1_00_000000, startTime=1452784904800, finishTime=1452784904859, timeTaken=59, status=SUCCEEDED, successfulAttemptID=attempt_1452784904563_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452784904849, Process_Duration=6, Process_End_Time=1452784904855, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-01-14 15:21:44,860 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784904563_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:44,861 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784904563_0001_1_00 [A4C743B7D843406AB05846ACCAAD9D35] : 1
2016-01-14 15:21:44,861 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784904563_0001_1_00 [A4C743B7D843406AB05846ACCAAD9D35]
2016-01-14 15:21:44,862 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:44,863 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784904563_0001_1][Event:VERTEX_FINISHED]: vertexName=A4C743B7D843406AB05846ACCAAD9D35, vertexId=vertex_1452784904563_0001_1_00, initRequestedTime=1452784904761, initedTime=1452784904777, startRequestedTime=1452784904778, startedTime=1452784904778, finishTime=1452784904861, timeTaken=83, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452784904849, Process_Duration=6, Process_End_Time=1452784904855, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10, vertexStats=firstTaskStartTime=1452784904800, firstTasksToStart=[ task_1452784904563_0001_1_00_000000 ], lastTaskFinishTime=1452784904857, lastTasksToFinish=[ task_1452784904563_0001_1_00_000000 ], minTaskDuration=57, maxTaskDuration=57, avgTaskDuration=57.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784904563_0001_1_00_000000 ], longestDurationTasks=[ task_1452784904563_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:44,863 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784904563_0001_1_00 [A4C743B7D843406AB05846ACCAAD9D35] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:44,863 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784904563_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:44,864 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784904563_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784904563_0001_00_000001, stoppedTime=1452784904864, exitStatus=0
2016-01-14 15:21:44,864 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:44,864 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784904563_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:44,865 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784904563_0001_1_00 [A4C743B7D843406AB05846ACCAAD9D35] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-01-14 15:21:44,865 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452784904563_0001_1
2016-01-14 15:21:44,866 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784904563_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452784904563_0001_1
2016-01-14 15:21:44,866 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: DAFB38CED3E84E29B4AB94E1F67EFD6E for vertex: vertex_1452784904563_0001_1_00 [A4C743B7D843406AB05846ACCAAD9D35]
2016-01-14 15:21:44,869 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784904563_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:21:44,869 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784904563_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:44,870 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784904563_0001_00_000001, Exiting
2016-01-14 15:21:44,870 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784904563_0001_00_000001
2016-01-14 15:21:44,870 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784904563_0001_00_000001
2016-01-14 15:21:44,872 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784904563_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:44,872 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784904563_0001_00_000001 completed successfully
2016-01-14 15:21:44,877 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452784904563_0001_1, queueSize=0
2016-01-14 15:21:44,878 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:21:44,879 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784904563_0001_1][Event:DAG_FINISHED]: dagId=dag_1452784904563_0001_1, startTime=1452784904760, finishTime=1452784904875, timeTaken=115, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452784904849, Process_Duration=6, Process_End_Time=1452784904855, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-01-14 15:21:44,879 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452784904563_0001_1 finished with state: SUCCEEDED
2016-01-14 15:21:44,879 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784904563_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:21:44,880 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452784904563_0001_00_000001
2016-01-14 15:21:44,880 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784904563_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:21:44 Completed Dag: dag_1452784904563_0001_1
2016-01-14 15:21:44,881 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452784904563_0001_1, dagState=SUCCEEDED
2016-01-14 15:21:44,882 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784904563_0001_1_post
2016-01-14 15:21:44,882 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:44,882 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:21:44,888 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...pesplatform/testSwap/swap, applicationId=application_1452784904563_0001
2016-01-14 15:21:44,890 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:21:44,890 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:21:44,891 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:21:44,891 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:21:44,891 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:21:44,891 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:21:44,892 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:21:44,892 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:21:44,892 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:21:44,893 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:21:44,893 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:21:44,894 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:21:44,894 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:21:44,900 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A1B8C261B8DB4B1F9D9AAAD4B39AC09D/F31FEFAA28AB485F81600DD8960507F0/.tez/application_1452784904563_0001
2016-01-14 15:21:44,900 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:00.359
2016-01-14 15:21:44,939 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSwap/swap
2016-01-14 15:21:44,971 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:21:44,971 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:21:45,016 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-01-14 15:21:45,025 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.024
2016-01-14 15:21:45,026 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 15:21:45,026 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 15:21:45,027 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:21:45,045 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower+upper.txt
2016-01-14 15:21:45,046 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped
2016-01-14 15:21:45,060 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 15:21:45,061 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower+upper.txt"]
2016-01-14 15:21:45,061 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped"]
2016-01-14 15:21:45,062 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 15:21:45,062 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 15:21:45,062 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 15:21:45,063 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...orm/testUnGroup/ungrouped
2016-01-14 15:21:45,063 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:21:45,078 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 15:21:45,083 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:45,084 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:21:45,084 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:21:45,093 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7FFAAFC6163D47CCA256C2D3161E595A/CDD9EDE0E84844EE9CAF950F2527CFD7/.tez/application_1452784905083_0001 doesn't exist and is created
2016-01-14 15:21:45,111 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:21:45,111 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:21:45,112 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7FFAAFC6163D47CCA256C2D3161E595A/CDD9EDE0E84844EE9CAF950F2527CFD7/.tez/application_1452784905083_0001
2016-01-14 15:21:45,116 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452784905083_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:45,129 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:21:45,130 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:21:45,132 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:21:45,132 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:21:45,133 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7FFAAFC6163D47CCA256C2D3161E595A/CDD9EDE0E84844EE9CAF950F2527CFD7/.tez/application_1452784905083_0001/localmode-log-dir
2016-01-14 15:21:45,133 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7FFAAFC6163D47CCA256C2D3161E595A/CDD9EDE0E84844EE9CAF950F2527CFD7/.tez/application_1452784905083_0001/localmode-log-dir/history.txt.appattempt_1452784905083_0001_000000, maxErrors=10
2016-01-14 15:21:45,133 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:21:45,134 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452784905083_0001_000000, appSubmitTime=1452784905116, launchTime=1452784905116
2016-01-14 15:21:45,141 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:21:45,143 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:21:45,148 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:21:45,148 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:21:45,152 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:60832
2016-01-14 15:21:45,154 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:21:45,156 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452784905083_0001_000000, startTime=1452784905156
2016-01-14 15:21:45,156 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:21:45,157 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:21:45,211 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:21:45,212 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:21:45,212 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...orm/testUnGroup/ungrouped, applicationId=application_1452784905083_0001, dagName=[7FFAAFC6163D47CCA256C2D3161E595A/CDD9EDE0E84844EE9CAF950F2527CFD7] test/(1/1) ...orm/testUnGroup/ungrouped
2016-01-14 15:21:45,243 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [7FFAAFC6163D47CCA256C2D3161E595A/CDD9EDE0E84844EE9CAF950F2527CFD7] test/(1/1) ...orm/testUnGroup/ungrouped
2016-01-14 15:21:45,244 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452784905083_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7FFAAFC6163D47CCA256C2D3161E595A/CDD9EDE0E84844EE9CAF950F2527CFD7/.tez/application_1452784905083_0001/localmode-log-dir/dag_1452784905083_0001_1.dot
2016-01-14 15:21:45,245 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7FFAAFC6163D47CCA256C2D3161E595A/CDD9EDE0E84844EE9CAF950F2527CFD7/.tez/application_1452784905083_0001/localmode-log-dir/dag_1452784905083_0001_1-tez-dag.pb.txt
2016-01-14 15:21:45,273 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784905083_0001_1
2016-01-14 15:21:45,274 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:45,274 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [7FFAAFC6163D47CCA256C2D3161E595A/CDD9EDE0E84844EE9CAF950F2527CFD7] test/(1/1) ...orm/testUnGroup/ungrouped
2016-01-14 15:21:45 Running Dag: dag_1452784905083_0001_1
2016-01-14 15:21:45,286 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905083_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452784905083_0001_1, submitTime=1452784905243
2016-01-14 15:21:45,286 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:21:45,286 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A73092230F1C4ABAAB0A1EB5B384630F
2016-01-14 15:21:45,287 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:21:45,287 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905083_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452784905083_0001_1, initTime=1452784905286
2016-01-14 15:21:45,287 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:21:45,288 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784905083_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:21:45,288 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905083_0001_1][Event:DAG_STARTED]: dagID=dag_1452784905083_0001_1, startTime=1452784905288
2016-01-14 15:21:45,289 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:21:45,288 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:21:45,289 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784905083_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:21:45,289 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A73092230F1C4ABAAB0A1EB5B384630F : {DF96F8EB0C6A4ADB98F86C3DB0752BE8={InputName=DF96F8EB0C6A4ADB98F86C3DB0752BE8}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:21:45,290 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: DF96F8EB0C6A4ADB98F86C3DB0752BE8, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:21:45,291 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452784905083_0001_1_00 [A73092230F1C4ABAAB0A1EB5B384630F]
2016-01-14 15:21:45,291 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452784905083_0001_1_00 [A73092230F1C4ABAAB0A1EB5B384630F]
2016-01-14 15:21:45,291 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452784905083_0001_1_00 [A73092230F1C4ABAAB0A1EB5B384630F]
2016-01-14 15:21:45,291 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452784905083_0001_1_00 [A73092230F1C4ABAAB0A1EB5B384630F]. Starting root input initializers: 1
2016-01-14 15:21:45,291 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...orm/testUnGroup/ungrouped, applicationId=application_1452784905083_0001, dagName=[7FFAAFC6163D47CCA256C2D3161E595A/CDD9EDE0E84844EE9CAF950F2527CFD7] test/(1/1) ...orm/testUnGroup/ungrouped
2016-01-14 15:21:45,293 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784905083_0001_1_00 [A73092230F1C4ABAAB0A1EB5B384630F] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:21:45,293 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: DF96F8EB0C6A4ADB98F86C3DB0752BE8 on vertex vertex_1452784905083_0001_1_00 [A73092230F1C4ABAAB0A1EB5B384630F]
2016-01-14 15:21:45,296 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:21:45,297 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input DF96F8EB0C6A4ADB98F86C3DB0752BE8 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:21:45,298 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:21:45,299 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:21:45,303 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 412
2016-01-14 15:21:45,304 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: DF96F8EB0C6A4ADB98F86C3DB0752BE8 on vertex vertex_1452784905083_0001_1_00 [A73092230F1C4ABAAB0A1EB5B384630F]
2016-01-14 15:21:45,304 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784905083_0001_1_00 [A73092230F1C4ABAAB0A1EB5B384630F] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:45,305 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {DF96F8EB0C6A4ADB98F86C3DB0752BE8=forAllWorkUnits=true, update=[1]}
2016-01-14 15:21:45,305 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452784905083_0001_1_00 [A73092230F1C4ABAAB0A1EB5B384630F] parallelism set to 1
2016-01-14 15:21:45,305 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784905083_0001_1_00 [A73092230F1C4ABAAB0A1EB5B384630F] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:45,305 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905083_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784905083_0001_1_00, eventCount=1
2016-01-14 15:21:45,306 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452784905083_0001_1_00 [A73092230F1C4ABAAB0A1EB5B384630F]
2016-01-14 15:21:45,306 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=F07BEB46ADC34C4D8CFE9FA0EF456A34, vertexId=vertex_1452784905083_0001_1_00 [A73092230F1C4ABAAB0A1EB5B384630F], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:21:45,309 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=F07BEB46ADC34C4D8CFE9FA0EF456A34, vertexId=vertex_1452784905083_0001_1_00 [A73092230F1C4ABAAB0A1EB5B384630F]
2016-01-14 15:21:45,311 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=F07BEB46ADC34C4D8CFE9FA0EF456A34, vertexName=A73092230F1C4ABAAB0A1EB5B384630F, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:21:45,312 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=F07BEB46ADC34C4D8CFE9FA0EF456A34, vertexId=vertex_1452784905083_0001_1_00 [A73092230F1C4ABAAB0A1EB5B384630F]
2016-01-14 15:21:45,313 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:45,314 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905083_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A73092230F1C4ABAAB0A1EB5B384630F, vertexId=vertex_1452784905083_0001_1_00, initRequestedTime=1452784905289, initedTime=1452784905313, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:21:45,314 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784905083_0001_1_00 [A73092230F1C4ABAAB0A1EB5B384630F] with distanceFromRoot: 0
2016-01-14 15:21:45,314 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784905083_0001_1_00 [A73092230F1C4ABAAB0A1EB5B384630F] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:21:45,315 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A73092230F1C4ABAAB0A1EB5B384630F
2016-01-14 15:21:45,315 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784905083_0001_1_00 [A73092230F1C4ABAAB0A1EB5B384630F]
2016-01-14 15:21:45,316 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:45,316 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905083_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784905083_0001_1_00, startRequestedTime=1452784905314, startedTime=1452784905315
2016-01-14 15:21:45,316 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784905083_0001_1_00 [A73092230F1C4ABAAB0A1EB5B384630F] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:45,317 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:45,318 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905083_0001_1][Event:TASK_STARTED]: vertexName=A73092230F1C4ABAAB0A1EB5B384630F, taskId=task_1452784905083_0001_1_00_000000, scheduledTime=1452784905317, launchTime=1452784905317
2016-01-14 15:21:45,318 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784905083_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:45,318 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784905083_0001_1_00 [A73092230F1C4ABAAB0A1EB5B384630F]
2016-01-14 15:21:45,319 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [7FFAAFC6163D47CCA256C2D3161E595A/CDD9EDE0E84844EE9CAF950F2527CFD7] test/(1/1) ...orm/testUnGroup/ungrouped, VertexName: A73092230F1C4ABAAB0A1EB5B384630F, VertexParallelism: 1, TaskAttemptID:attempt_1452784905083_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DF96F8EB0C6A4ADB98F86C3DB0752BE8, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F07BEB46ADC34C4D8CFE9FA0EF456A34, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:45,319 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784905083_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:45,319 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:45,321 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:21:45,322 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452784905083_0001, with dag id: dag_1452784905083_0001_1
2016-01-14 15:21:45,324 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:21:45,325 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:21:45,328 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784905083_0001_00_000001
2016-01-14 15:21:45,329 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784905083_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:45,332 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784905083_0001_00_000001
2016-01-14 15:21:45,333 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:21:45,335 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784905083_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:45,336 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:45,336 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905083_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784905083_0001_00_000001, launchTime=1452784905335
2016-01-14 15:21:45,338 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784905083_0001_00_000001
2016-01-14 15:21:45,338 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784905083_0001_1_00_000000_0] to container: [container_1452784905083_0001_00_000001]
2016-01-14 15:21:45,339 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784905083_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:45,339 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784905083_0001_1_00_000000_0] started. Is using containerId: [container_1452784905083_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:21:45,339 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905083_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A73092230F1C4ABAAB0A1EB5B384630F, taskAttemptId=attempt_1452784905083_0001_1_00_000000_0, startTime=1452784905339, containerId=container_1452784905083_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784905083_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:21:45,340 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784905083_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:45,340 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:45,342 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784905083_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:45,342 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784905083_0001_00_000001 given task: attempt_1452784905083_0001_1_00_000000_0
2016-01-14 15:21:45,343 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784905083_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784905083_0001_1_00_000000_0
2016-01-14 15:21:45 Starting to run new task attempt: attempt_1452784905083_0001_1_00_000000_0
2016-01-14 15:21:45,346 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784905083_0001_1_00_000000_0
2016-01-14 15:21:45,346 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:45,347 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:45,347 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:45,348 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:45,348 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:45,348 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:45,349 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7FFAAFC6163D47CCA256C2D3161E595A/CDD9EDE0E84844EE9CAF950F2527CFD7] test/(1/1) ...orm/testUnGroup/ungrouped, VertexName: A73092230F1C4ABAAB0A1EB5B384630F, VertexParallelism: 1, TaskAttemptID:attempt_1452784905083_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DF96F8EB0C6A4ADB98F86C3DB0752BE8, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F07BEB46ADC34C4D8CFE9FA0EF456A34, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:45,350 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:45,350 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784905083_0001_1_00_000000_0
2016-01-14 15:21:45,351 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:45,351 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:45,352 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:45,352 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F07BEB46ADC34C4D8CFE9FA0EF456A34, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:21:45,353 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:45,353 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F07BEB46ADC34C4D8CFE9FA0EF456A34
2016-01-14 15:21:45,354 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:21:45,354 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: F07BEB46ADC34C4D8CFE9FA0EF456A34
2016-01-14 15:21:45,354 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=DF96F8EB0C6A4ADB98F86C3DB0752BE8, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:21:45,356 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:45,357 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784905083_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:21:45,359 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:45,361 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: DF96F8EB0C6A4ADB98F86C3DB0752BE8
2016-01-14 15:21:45,361 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: DF96F8EB0C6A4ADB98F86C3DB0752BE8
2016-01-14 15:21:45,369 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:21:45,369 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: DF96F8EB0C6A4ADB98F86C3DB0752BE8
2016-01-14 15:21:45,370 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A73092230F1C4ABAAB0A1EB5B384630F, ordinal: 0
2016-01-14 15:21:45,370 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A73092230F1C4ABAAB0A1EB5B384630F, mem on start (mb), free: 368, total: 404, max: 672
2016-01-14 15:21:45,370 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:45,370 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:45,370 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: DF96F8EB0C6A4ADB98F86C3DB0752BE8
2016-01-14 15:21:45,370 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:45,375 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:21:45,375 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F07BEB46ADC34C4D8CFE9FA0EF456A34
2016-01-14 15:21:45,376 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:45,376 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:45,376 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:21:45,376 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:45,377 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:21:45,377 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:45,377 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:21:45,377 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-01-14 15:21:45,378 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F07BEB46ADC34C4D8CFE9FA0EF456A34, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:21:45,378 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, DF96F8EB0C6A4ADB98F86C3DB0752BE8, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:21:45,378 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:45,379 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: DF96F8EB0C6A4ADB98F86C3DB0752BE8 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:45,379 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:21:45,379 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:45,379 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: DF96F8EB0C6A4ADB98F86C3DB0752BE8
2016-01-14 15:21:45,380 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:45,380 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: DF96F8EB0C6A4ADB98F86C3DB0752BE8
2016-01-14 15:21:45,381 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:45,381 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784905083_0001_1_00_000000_0
2016-01-14 15:21:45,381 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:21:45,383 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@716d7daf
2016-01-14 15:21:45,384 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@716d7daf
2016-01-14 15:21:45,384 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:21:45,384 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:21:45,385 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower+upper.txt"]
2016-01-14 15:21:45,387 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower+upper.txt"] streamed: true, id: DF96F8EB0C6A4ADB98F86C3DB0752BE8
2016-01-14 15:21:45,387 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped"], id: F07BEB46ADC34C4D8CFE9FA0EF456A34
2016-01-14 15:21:45,388 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped"]
2016-01-14 15:21:45,389 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower+upper.txt"]
2016-01-14 15:21:45,389 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:45,389 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A73092230F1C4ABAAB0A1EB5B384630F, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:21:45,392 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:21:45,394 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:21:45,395 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452784905083_0001_1_00_000000_0
2016-01-14 15:21:45,396 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452784905083_0001_1_00_000000_0 given a go for committing the task output.
2016-01-14 15:21:45,397 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784905083_0001_1_00_000000_0
2016-01-14 15:21:45,397 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A73092230F1C4ABAAB0A1EB5B384630F, mem on close (mb), free: 367, total: 404, max: 672
2016-01-14 15:21:45 Completed running task attempt: attempt_1452784905083_0001_1_00_000000_0
2016-01-14 15:21:45,398 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784905083_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:45,401 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905083_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A73092230F1C4ABAAB0A1EB5B384630F, taskAttemptId=attempt_1452784905083_0001_1_00_000000_0, startTime=1452784905339, finishTime=1452784905400, timeTaken=61, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=423624704, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452784905390, Process_Duration=6, Process_End_Time=1452784905396, Read_Duration=1, Tuples_Read=5, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10
2016-01-14 15:21:45,401 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:45,401 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784905083_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:45,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=423624704, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10][cascading.flow.SliceCounters Process_Begin_Time=1452784905390, Process_Duration=6, Process_End_Time=1452784905396, Read_Duration=1, Tuples_Read=5, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5, Tuples_Written=10]]
2016-01-14 15:21:45,401 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784905083_0001_1_00_000000_0
2016-01-14 15:21:45,401 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:45,401 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:45,401 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784905083_0001_00_000001
2016-01-14 15:21:45,402 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:45,402 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905083_0001_1][Event:TASK_FINISHED]: vertexName=A73092230F1C4ABAAB0A1EB5B384630F, taskId=task_1452784905083_0001_1_00_000000, startTime=1452784905339, finishTime=1452784905401, timeTaken=62, status=SUCCEEDED, successfulAttemptID=attempt_1452784905083_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=423624704, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452784905390, Process_Duration=6, Process_End_Time=1452784905396, Read_Duration=1, Tuples_Read=5, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10
2016-01-14 15:21:45,402 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784905083_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:45,402 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784905083_0001_1_00 [A73092230F1C4ABAAB0A1EB5B384630F] : 1
2016-01-14 15:21:45,403 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784905083_0001_1_00 [A73092230F1C4ABAAB0A1EB5B384630F]
2016-01-14 15:21:45,403 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:45,405 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:45,405 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905083_0001_1][Event:VERTEX_FINISHED]: vertexName=A73092230F1C4ABAAB0A1EB5B384630F, vertexId=vertex_1452784905083_0001_1_00, initRequestedTime=1452784905289, initedTime=1452784905313, startRequestedTime=1452784905314, startedTime=1452784905315, finishTime=1452784905403, timeTaken=88, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=423624704, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452784905390, Process_Duration=6, Process_End_Time=1452784905396, Read_Duration=1, Tuples_Read=5, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10, vertexStats=firstTaskStartTime=1452784905339, firstTasksToStart=[ task_1452784905083_0001_1_00_000000 ], lastTaskFinishTime=1452784905400, lastTasksToFinish=[ task_1452784905083_0001_1_00_000000 ], minTaskDuration=61, maxTaskDuration=61, avgTaskDuration=61.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784905083_0001_1_00_000000 ], longestDurationTasks=[ task_1452784905083_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:45,405 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784905083_0001_1_00 [A73092230F1C4ABAAB0A1EB5B384630F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:45,405 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784905083_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:45,405 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905083_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784905083_0001_00_000001, stoppedTime=1452784905405, exitStatus=0
2016-01-14 15:21:45,406 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784905083_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:45,406 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:45,406 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784905083_0001_1_00 [A73092230F1C4ABAAB0A1EB5B384630F] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-01-14 15:21:45,406 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452784905083_0001_1
2016-01-14 15:21:45,406 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905083_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452784905083_0001_1
2016-01-14 15:21:45,407 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: F07BEB46ADC34C4D8CFE9FA0EF456A34 for vertex: vertex_1452784905083_0001_1_00 [A73092230F1C4ABAAB0A1EB5B384630F]
2016-01-14 15:21:45,413 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784905083_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:21:45,413 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784905083_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:45,413 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784905083_0001_00_000001, Exiting
2016-01-14 15:21:45,413 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784905083_0001_00_000001
2016-01-14 15:21:45,413 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784905083_0001_00_000001
2016-01-14 15:21:45,415 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784905083_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:45,415 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784905083_0001_00_000001 completed successfully
2016-01-14 15:21:45,420 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452784905083_0001_1, queueSize=0
2016-01-14 15:21:45,421 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:21:45,421 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905083_0001_1][Event:DAG_FINISHED]: dagId=dag_1452784905083_0001_1, startTime=1452784905288, finishTime=1452784905417, timeTaken=129, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=423624704, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452784905390, Process_Duration=6, Process_End_Time=1452784905396, Read_Duration=1, Tuples_Read=5, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10
2016-01-14 15:21:45,421 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452784905083_0001_1 finished with state: SUCCEEDED
2016-01-14 15:21:45,421 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784905083_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:21:45,422 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452784905083_0001_00_000001
2016-01-14 15:21:45,422 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784905083_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:21:45 Completed Dag: dag_1452784905083_0001_1
2016-01-14 15:21:45,423 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452784905083_0001_1, dagState=SUCCEEDED
2016-01-14 15:21:45,423 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784905083_0001_1_post
2016-01-14 15:21:45,423 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:45,423 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:21:45,429 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...orm/testUnGroup/ungrouped, applicationId=application_1452784905083_0001
2016-01-14 15:21:45,432 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:21:45,432 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:21:45,432 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:21:45,433 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:21:45,433 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:21:45,434 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:21:45,434 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:21:45,434 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:21:45,435 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:21:45,435 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:21:45,435 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:21:45,436 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:21:45,436 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:21:45,441 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7FFAAFC6163D47CCA256C2D3161E595A/CDD9EDE0E84844EE9CAF950F2527CFD7/.tez/application_1452784905083_0001
2016-01-14 15:21:45,441 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:00.381
2016-01-14 15:21:45,478 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped
2016-01-14 15:21:45,509 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:21:45,509 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:21:45,562 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.024
2016-01-14 15:21:45,577 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.039
2016-01-14 15:21:45,578 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-01-14 15:21:45,579 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-01-14 15:21:45,579 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:21:45,598 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt
2016-01-14 15:21:45,613 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt
2016-01-14 15:21:45,624 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown
2016-01-14 15:21:45,694 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] starting
2016-01-14 15:21:45,694 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-01-14 15:21:45,694 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 15:21:45,694 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown"]
2016-01-14 15:21:45,694 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] parallel execution of steps is enabled: false
2016-01-14 15:21:45,694 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] executing total steps: 1
2016-01-14 15:21:45,695 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] allocating management threads: 1
2016-01-14 15:21:45,695 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] starting step: (1/1) ...lexcogrouprelativeunknown
2016-01-14 15:21:45,695 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [lower*upper] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:21:45,714 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] tez session mode enabled: true
2016-01-14 15:21:45,721 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:45,722 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:21:45,722 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:21:45,731 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A92072F8C7B04072847C4ED977E8C48B/B9BEEE83742C461BAEBD9AA0318B3850/.tez/application_1452784905721_0001 doesn't exist and is created
2016-01-14 15:21:45,748 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:21:45,748 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:21:45,749 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A92072F8C7B04072847C4ED977E8C48B/B9BEEE83742C461BAEBD9AA0318B3850/.tez/application_1452784905721_0001
2016-01-14 15:21:45,754 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452784905721_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:45,766 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:21:45,767 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:21:45,769 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:21:45,769 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:21:45,769 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A92072F8C7B04072847C4ED977E8C48B/B9BEEE83742C461BAEBD9AA0318B3850/.tez/application_1452784905721_0001/localmode-log-dir
2016-01-14 15:21:45,769 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A92072F8C7B04072847C4ED977E8C48B/B9BEEE83742C461BAEBD9AA0318B3850/.tez/application_1452784905721_0001/localmode-log-dir/history.txt.appattempt_1452784905721_0001_000000, maxErrors=10
2016-01-14 15:21:45,770 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:21:45,770 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452784905721_0001_000000, appSubmitTime=1452784905754, launchTime=1452784905754
2016-01-14 15:21:45,775 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:21:45,778 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:21:45,779 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:21:45,779 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:21:45,782 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:60673
2016-01-14 15:21:45,784 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:21:45,787 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452784905721_0001_000000, startTime=1452784905787
2016-01-14 15:21:45,787 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:21:45,787 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:21:45,848 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:21:45,848 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:21:45,849 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lexcogrouprelativeunknown, applicationId=application_1452784905721_0001, dagName=[A92072F8C7B04072847C4ED977E8C48B/B9BEEE83742C461BAEBD9AA0318B3850] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-01-14 15:21:45,874 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [A92072F8C7B04072847C4ED977E8C48B/B9BEEE83742C461BAEBD9AA0318B3850] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-01-14 15:21:45,876 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452784905721_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A92072F8C7B04072847C4ED977E8C48B/B9BEEE83742C461BAEBD9AA0318B3850/.tez/application_1452784905721_0001/localmode-log-dir/dag_1452784905721_0001_1.dot
2016-01-14 15:21:45,878 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A92072F8C7B04072847C4ED977E8C48B/B9BEEE83742C461BAEBD9AA0318B3850/.tez/application_1452784905721_0001/localmode-log-dir/dag_1452784905721_0001_1-tez-dag.pb.txt
2016-01-14 15:21:45,900 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784905721_0001_1
2016-01-14 15:21:45,900 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:45,900 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [A92072F8C7B04072847C4ED977E8C48B/B9BEEE83742C461BAEBD9AA0318B3850] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-01-14 15:21:45 Running Dag: dag_1452784905721_0001_1
2016-01-14 15:21:45,916 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452784905721_0001_1, submitTime=1452784905874
2016-01-14 15:21:45,916 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:21:45,917 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex EBCB27F1E25243EAB2D9E226D90BE526
2016-01-14 15:21:45,918 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:21:45,919 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452784905721_0001_1, initTime=1452784905916
2016-01-14 15:21:45,919 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:21:45,919 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784905721_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:21:45,921 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:21:45,921 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:DAG_STARTED]: dagID=dag_1452784905721_0001_1, startTime=1452784905920
2016-01-14 15:21:45,921 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:21:45,922 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784905721_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:21:45,922 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F21901D11B07417BA7F4F6C02C4B80F1 : {CAE3EFD220354A02A1C4BBA2AA40D42F={InputName=CAE3EFD220354A02A1C4BBA2AA40D42F}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:21:45,922 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: CAE3EFD220354A02A1C4BBA2AA40D42F, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:21:45,922 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452784905721_0001_1_00 [F21901D11B07417BA7F4F6C02C4B80F1]
2016-01-14 15:21:45,922 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452784905721_0001_1_00 [F21901D11B07417BA7F4F6C02C4B80F1]
2016-01-14 15:21:45,922 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452784905721_0001_1_00 [F21901D11B07417BA7F4F6C02C4B80F1]
2016-01-14 15:21:45,923 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452784905721_0001_1_00 [F21901D11B07417BA7F4F6C02C4B80F1]. Starting root input initializers: 1
2016-01-14 15:21:45,923 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lexcogrouprelativeunknown, applicationId=application_1452784905721_0001, dagName=[A92072F8C7B04072847C4ED977E8C48B/B9BEEE83742C461BAEBD9AA0318B3850] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-01-14 15:21:45,924 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784905721_0001_1_00 [F21901D11B07417BA7F4F6C02C4B80F1] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:21:45,924 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E3D634B224BA49CDA9B6F031ABA527A2 : {74B77BF0AD544FBEA470B56334FEEBC2={InputName=74B77BF0AD544FBEA470B56334FEEBC2}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:21:45,924 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 74B77BF0AD544FBEA470B56334FEEBC2, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:21:45,924 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452784905721_0001_1_01 [E3D634B224BA49CDA9B6F031ABA527A2]
2016-01-14 15:21:45,924 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452784905721_0001_1_01 [E3D634B224BA49CDA9B6F031ABA527A2]
2016-01-14 15:21:45,924 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452784905721_0001_1_01 [E3D634B224BA49CDA9B6F031ABA527A2]
2016-01-14 15:21:45,925 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452784905721_0001_1_01 [E3D634B224BA49CDA9B6F031ABA527A2]. Starting root input initializers: 1
2016-01-14 15:21:45,924 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: CAE3EFD220354A02A1C4BBA2AA40D42F on vertex vertex_1452784905721_0001_1_00 [F21901D11B07417BA7F4F6C02C4B80F1]
2016-01-14 15:21:45,930 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:21:45,930 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input CAE3EFD220354A02A1C4BBA2AA40D42F asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:21:45,931 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:21:45,932 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:21:45,935 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-01-14 15:21:45,935 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784905721_0001_1_01 [E3D634B224BA49CDA9B6F031ABA527A2] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:21:45,936 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452784905721_0001_1_02 [EBCB27F1E25243EAB2D9E226D90BE526]
2016-01-14 15:21:45,938 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: CAE3EFD220354A02A1C4BBA2AA40D42F on vertex vertex_1452784905721_0001_1_00 [F21901D11B07417BA7F4F6C02C4B80F1]
2016-01-14 15:21:45,942 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:21:45,942 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452784905721_0001_1_02 [EBCB27F1E25243EAB2D9E226D90BE526]
2016-01-14 15:21:45,942 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452784905721_0001_1_02 [EBCB27F1E25243EAB2D9E226D90BE526]
2016-01-14 15:21:45,942 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452784905721_0001_1_02 [EBCB27F1E25243EAB2D9E226D90BE526]
2016-01-14 15:21:45,942 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=361A3BFB47874712A1226A1DD07D1243, vertexId=vertex_1452784905721_0001_1_02 [EBCB27F1E25243EAB2D9E226D90BE526], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:21:45,942 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=361A3BFB47874712A1226A1DD07D1243, vertexId=vertex_1452784905721_0001_1_02 [EBCB27F1E25243EAB2D9E226D90BE526]
2016-01-14 15:21:45,943 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] submitted tez dag to app master: application_1452784905721_0001, with dag id: dag_1452784905721_0001_1
2016-01-14 15:21:45,946 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 74B77BF0AD544FBEA470B56334FEEBC2 on vertex vertex_1452784905721_0001_1_01 [E3D634B224BA49CDA9B6F031ABA527A2]
2016-01-14 15:21:45,946 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=361A3BFB47874712A1226A1DD07D1243, vertexName=EBCB27F1E25243EAB2D9E226D90BE526, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:21:45,947 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=361A3BFB47874712A1226A1DD07D1243, vertexId=vertex_1452784905721_0001_1_02 [EBCB27F1E25243EAB2D9E226D90BE526]
2016-01-14 15:21:45,948 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:21:45,948 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:45,949 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EBCB27F1E25243EAB2D9E226D90BE526, vertexId=vertex_1452784905721_0001_1_02, initRequestedTime=1452784905936, initedTime=1452784905948, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:21:45,949 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784905721_0001_1_02 [EBCB27F1E25243EAB2D9E226D90BE526] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:21:45,949 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 74B77BF0AD544FBEA470B56334FEEBC2 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:21:45,949 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784905721_0001_1_00 [F21901D11B07417BA7F4F6C02C4B80F1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:45,950 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {CAE3EFD220354A02A1C4BBA2AA40D42F=forAllWorkUnits=true, update=[1]}
2016-01-14 15:21:45,950 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:21:45,950 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452784905721_0001_1_00 [F21901D11B07417BA7F4F6C02C4B80F1] parallelism set to 1
2016-01-14 15:21:45,951 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784905721_0001_1_00 [F21901D11B07417BA7F4F6C02C4B80F1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:45,953 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:21:45,954 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784905721_0001_1_00, eventCount=1
2016-01-14 15:21:45,955 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:45,956 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F21901D11B07417BA7F4F6C02C4B80F1, vertexId=vertex_1452784905721_0001_1_00, initRequestedTime=1452784905922, initedTime=1452784905955, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:21:45,956 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784905721_0001_1_00 [F21901D11B07417BA7F4F6C02C4B80F1] with distanceFromRoot: 0
2016-01-14 15:21:45,956 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784905721_0001_1_00 [F21901D11B07417BA7F4F6C02C4B80F1] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:21:45,957 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-01-14 15:21:45,958 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F21901D11B07417BA7F4F6C02C4B80F1
2016-01-14 15:21:45,958 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784905721_0001_1_00 [F21901D11B07417BA7F4F6C02C4B80F1]
2016-01-14 15:21:45,958 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 74B77BF0AD544FBEA470B56334FEEBC2 on vertex vertex_1452784905721_0001_1_01 [E3D634B224BA49CDA9B6F031ABA527A2]
2016-01-14 15:21:45,958 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784905721_0001_1_00, startRequestedTime=1452784905958, startedTime=1452784905958
2016-01-14 15:21:45,958 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:45,959 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784905721_0001_1_00 [F21901D11B07417BA7F4F6C02C4B80F1] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:45,960 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:45,960 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:TASK_STARTED]: vertexName=F21901D11B07417BA7F4F6C02C4B80F1, taskId=task_1452784905721_0001_1_00_000000, scheduledTime=1452784905959, launchTime=1452784905959
2016-01-14 15:21:45,960 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784905721_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:45,960 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784905721_0001_1_01 [E3D634B224BA49CDA9B6F031ABA527A2] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:45,960 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {74B77BF0AD544FBEA470B56334FEEBC2=forAllWorkUnits=true, update=[1]}
2016-01-14 15:21:45,961 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452784905721_0001_1_01 [E3D634B224BA49CDA9B6F031ABA527A2] parallelism set to 1
2016-01-14 15:21:45,961 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784905721_0001_1_01 [E3D634B224BA49CDA9B6F031ABA527A2] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:45,961 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784905721_0001_1_01, eventCount=1
2016-01-14 15:21:45,962 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:45,962 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E3D634B224BA49CDA9B6F031ABA527A2, vertexId=vertex_1452784905721_0001_1_01, initRequestedTime=1452784905924, initedTime=1452784905961, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:21:45,962 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784905721_0001_1_01 [E3D634B224BA49CDA9B6F031ABA527A2] with distanceFromRoot: 0
2016-01-14 15:21:45,963 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784905721_0001_1_01 [E3D634B224BA49CDA9B6F031ABA527A2] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:21:45,963 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452784905721_0001_1_00 for vertex: vertex_1452784905721_0001_1_02 [EBCB27F1E25243EAB2D9E226D90BE526] numStartedSources: 1 numSources: 2
2016-01-14 15:21:45,963 INFO impl.VertexImpl (VertexImpl.java:transition(3160)) - Cannot start vertex: vertex_1452784905721_0001_1_02 [EBCB27F1E25243EAB2D9E226D90BE526] numStartedSources: 1 numSources: 2
2016-01-14 15:21:45,963 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784905721_0001_1_00 [F21901D11B07417BA7F4F6C02C4B80F1]
2016-01-14 15:21:45,964 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E3D634B224BA49CDA9B6F031ABA527A2
2016-01-14 15:21:45,964 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784905721_0001_1_01 [E3D634B224BA49CDA9B6F031ABA527A2]
2016-01-14 15:21:45,967 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:45,967 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784905721_0001_1_01, startRequestedTime=1452784905964, startedTime=1452784905964
2016-01-14 15:21:45,967 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784905721_0001_1_01 [E3D634B224BA49CDA9B6F031ABA527A2] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:45,967 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A92072F8C7B04072847C4ED977E8C48B/B9BEEE83742C461BAEBD9AA0318B3850] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: F21901D11B07417BA7F4F6C02C4B80F1, VertexParallelism: 1, TaskAttemptID:attempt_1452784905721_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CAE3EFD220354A02A1C4BBA2AA40D42F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EBCB27F1E25243EAB2D9E226D90BE526, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:21:45,968 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784905721_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:45,969 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:TASK_STARTED]: vertexName=E3D634B224BA49CDA9B6F031ABA527A2, taskId=task_1452784905721_0001_1_01_000000, scheduledTime=1452784905968, launchTime=1452784905968
2016-01-14 15:21:45,969 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784905721_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:45,969 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452784905721_0001_1_01 for vertex: vertex_1452784905721_0001_1_02 [EBCB27F1E25243EAB2D9E226D90BE526] numStartedSources: 2 numSources: 2
2016-01-14 15:21:45,969 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:45,969 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784905721_0001_1_02 [EBCB27F1E25243EAB2D9E226D90BE526] with distanceFromRoot: 1
2016-01-14 15:21:45,970 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:45,970 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784905721_0001_1_01 [E3D634B224BA49CDA9B6F031ABA527A2]
2016-01-14 15:21:45,970 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E3D634B224BA49CDA9B6F031ABA527A2 in vertex: EBCB27F1E25243EAB2D9E226D90BE526
2016-01-14 15:21:45,970 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F21901D11B07417BA7F4F6C02C4B80F1 in vertex: EBCB27F1E25243EAB2D9E226D90BE526
2016-01-14 15:21:45,971 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: EBCB27F1E25243EAB2D9E226D90BE526 with 2 source tasks and 1 pending tasks
2016-01-14 15:21:45,971 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:21:45,971 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:45,971 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784905721_0001_1_02, startRequestedTime=1452784905970, startedTime=1452784905970
2016-01-14 15:21:45,972 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784905721_0001_1_02 [EBCB27F1E25243EAB2D9E226D90BE526] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:45,972 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A92072F8C7B04072847C4ED977E8C48B/B9BEEE83742C461BAEBD9AA0318B3850] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: E3D634B224BA49CDA9B6F031ABA527A2, VertexParallelism: 1, TaskAttemptID:attempt_1452784905721_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=74B77BF0AD544FBEA470B56334FEEBC2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EBCB27F1E25243EAB2D9E226D90BE526, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:21:45,973 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784905721_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:45,973 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:21:45,973 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:21:45,974 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784905721_0001_00_000001
2016-01-14 15:21:45,974 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784905721_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:45,974 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:45,976 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784905721_0001_00_000001
2016-01-14 15:21:45,976 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:21:45,977 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784905721_0001_00_000002
2016-01-14 15:21:45,977 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:45,978 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784905721_0001_00_000001, launchTime=1452784905977
2016-01-14 15:21:45,978 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784905721_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:45,979 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784905721_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:45,980 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784905721_0001_00_000001
2016-01-14 15:21:45,982 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784905721_0001_00_000002
2016-01-14 15:21:45,982 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:21:45,983 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784905721_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:45,983 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784905721_0001_00_000002, launchTime=1452784905983
2016-01-14 15:21:45,983 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:45,984 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784905721_0001_00_000002
2016-01-14 15:21:45,988 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784905721_0001_1_00_000000_0] to container: [container_1452784905721_0001_00_000001]
2016-01-14 15:21:45,988 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784905721_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:45,988 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784905721_0001_00_000001 given task: attempt_1452784905721_0001_1_00_000000_0
2016-01-14 15:21:45,989 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784905721_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784905721_0001_1_00_000000_0
2016-01-14 15:21:45,989 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784905721_0001_1_01_000000_0] to container: [container_1452784905721_0001_00_000002]
2016-01-14 15:21:45 Starting to run new task attempt: attempt_1452784905721_0001_1_00_000000_0
2016-01-14 15:21:45,989 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784905721_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:45,989 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784905721_0001_1_00_000000_0
2016-01-14 15:21:45,989 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784905721_0001_00_000002 given task: attempt_1452784905721_0001_1_01_000000_0
2016-01-14 15:21:45,990 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:45,990 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:45,990 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:45,990 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:45,990 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:45,990 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:45,991 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A92072F8C7B04072847C4ED977E8C48B/B9BEEE83742C461BAEBD9AA0318B3850] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: F21901D11B07417BA7F4F6C02C4B80F1, VertexParallelism: 1, TaskAttemptID:attempt_1452784905721_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CAE3EFD220354A02A1C4BBA2AA40D42F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EBCB27F1E25243EAB2D9E226D90BE526, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:21:45,991 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:45,992 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784905721_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784905721_0001_1_01_000000_0
2016-01-14 15:21:45,990 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784905721_0001_1_00_000000_0] started. Is using containerId: [container_1452784905721_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:21:45,992 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:45,992 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F21901D11B07417BA7F4F6C02C4B80F1, taskAttemptId=attempt_1452784905721_0001_1_00_000000_0, startTime=1452784905990, containerId=container_1452784905721_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784905721_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:21:45,993 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784905721_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:45,994 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784905721_0001_1_01_000000_0] started. Is using containerId: [container_1452784905721_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:21:45,994 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:45,994 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E3D634B224BA49CDA9B6F031ABA527A2, taskAttemptId=attempt_1452784905721_0001_1_01_000000_0, startTime=1452784905993, containerId=container_1452784905721_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784905721_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:21:45,999 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784905721_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:46,000 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784905721_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:46,000 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784905721_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:46 Starting to run new task attempt: attempt_1452784905721_0001_1_01_000000_0
2016-01-14 15:21:46,001 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784905721_0001_1_00_000000_0
2016-01-14 15:21:46,005 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784905721_0001_1_01_000000_0
2016-01-14 15:21:46,005 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784905721_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:21:46,005 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:46,005 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:46,006 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:46,006 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:46,006 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:46,006 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:46,006 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:46,006 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A92072F8C7B04072847C4ED977E8C48B/B9BEEE83742C461BAEBD9AA0318B3850] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: E3D634B224BA49CDA9B6F031ABA527A2, VertexParallelism: 1, TaskAttemptID:attempt_1452784905721_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=74B77BF0AD544FBEA470B56334FEEBC2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EBCB27F1E25243EAB2D9E226D90BE526, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:21:46,007 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:46,007 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:46,008 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784905721_0001_1_01_000000_0
2016-01-14 15:21:46,010 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:46,011 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:46,015 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:46,021 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=74B77BF0AD544FBEA470B56334FEEBC2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:21:46,022 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:46,022 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 74B77BF0AD544FBEA470B56334FEEBC2
2016-01-14 15:21:46,023 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:46,023 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 74B77BF0AD544FBEA470B56334FEEBC2
2016-01-14 15:21:46,026 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:46,029 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:46,030 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=CAE3EFD220354A02A1C4BBA2AA40D42F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:21:46,031 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:46,031 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: CAE3EFD220354A02A1C4BBA2AA40D42F
2016-01-14 15:21:46,031 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: CAE3EFD220354A02A1C4BBA2AA40D42F
2016-01-14 15:21:46,033 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784905721_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:21:46,035 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=EBCB27F1E25243EAB2D9E226D90BE526, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:21:46,036 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:46,036 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: EBCB27F1E25243EAB2D9E226D90BE526
2016-01-14 15:21:46,038 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:21:46,038 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: EBCB27F1E25243EAB2D9E226D90BE526
2016-01-14 15:21:46,040 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=EBCB27F1E25243EAB2D9E226D90BE526, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:21:46,040 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:46,040 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: EBCB27F1E25243EAB2D9E226D90BE526
2016-01-14 15:21:46,042 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F21901D11B07417BA7F4F6C02C4B80F1, ordinal: 0
2016-01-14 15:21:46,043 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F21901D11B07417BA7F4F6C02C4B80F1, mem on start (mb), free: 357, total: 404, max: 672
2016-01-14 15:21:46,043 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:46,044 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:46,042 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:21:46,044 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 74B77BF0AD544FBEA470B56334FEEBC2
2016-01-14 15:21:46,044 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 74B77BF0AD544FBEA470B56334FEEBC2
2016-01-14 15:21:46,044 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: EBCB27F1E25243EAB2D9E226D90BE526
2016-01-14 15:21:46,044 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:46,044 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:21:46,044 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:21:46,045 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: CAE3EFD220354A02A1C4BBA2AA40D42F
2016-01-14 15:21:46,045 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: CAE3EFD220354A02A1C4BBA2AA40D42F
2016-01-14 15:21:46,045 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: EBCB27F1E25243EAB2D9E226D90BE526
2016-01-14 15:21:46,045 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:46,050 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:46,050 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:21:46,050 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:46,050 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:21:46,050 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:21:46,051 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:21:46,051 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, CAE3EFD220354A02A1C4BBA2AA40D42F, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:21:46,051 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, EBCB27F1E25243EAB2D9E226D90BE526, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:21:46,051 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:46,051 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: CAE3EFD220354A02A1C4BBA2AA40D42F
2016-01-14 15:21:46,051 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:46,051 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: CAE3EFD220354A02A1C4BBA2AA40D42F
2016-01-14 15:21:46,051 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: CAE3EFD220354A02A1C4BBA2AA40D42F being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:46,052 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:21:46,052 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:46,052 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:46,052 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: EBCB27F1E25243EAB2D9E226D90BE526
2016-01-14 15:21:46,055 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784905721_0001_1_00_000000_0
2016-01-14 15:21:46,055 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E3D634B224BA49CDA9B6F031ABA527A2, ordinal: 1
2016-01-14 15:21:46,055 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E3D634B224BA49CDA9B6F031ABA527A2, mem on start (mb), free: 356, total: 404, max: 672
2016-01-14 15:21:46,056 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:46,056 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:46,056 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:46,056 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:46,057 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:46,057 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:21:46,057 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:46,058 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:21:46,058 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:21:46,058 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:21:46,058 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-01-14 15:21:46,060 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2297933
2016-01-14 15:21:46,059 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:21:46,060 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 74B77BF0AD544FBEA470B56334FEEBC2, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:21:46,061 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, EBCB27F1E25243EAB2D9E226D90BE526, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:21:46,062 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:46,063 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 74B77BF0AD544FBEA470B56334FEEBC2 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:46,063 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:21:46,063 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 74B77BF0AD544FBEA470B56334FEEBC2
2016-01-14 15:21:46,064 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:46,064 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 74B77BF0AD544FBEA470B56334FEEBC2
2016-01-14 15:21:46,064 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:46,065 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:46,065 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2297933
2016-01-14 15:21:46,065 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:21:46,065 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:21:46,065 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784905721_0001_1_01_000000_0
2016-01-14 15:21:46,066 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:21:46,066 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3ea1d9f0
2016-01-14 15:21:46,067 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3ea1d9f0
2016-01-14 15:21:46,067 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:21:46,068 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:21:46,075 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 15:21:46,076 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"] streamed: true, id: 74B77BF0AD544FBEA470B56334FEEBC2
2016-01-14 15:21:46,076 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]], id: 300594D316AC4B119BDC92ADD1A3FAEC
2016-01-14 15:21:46,076 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] 300594D316AC4B119BDC92ADD1A3FAEC
2016-01-14 15:21:46,077 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@c24ea6a; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@4b99db60; comparator=cascading.tuple.hadoop.util.TupleComparator@73bd08d6; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:21:46,082 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:21:46,083 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:21:46,083 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:21:46,088 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt"] streamed: true, id: CAE3EFD220354A02A1C4BBA2AA40D42F
2016-01-14 15:21:46,089 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]], id: 300594D316AC4B119BDC92ADD1A3FAEC
2016-01-14 15:21:46,089 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] 300594D316AC4B119BDC92ADD1A3FAEC
2016-01-14 15:21:46,090 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@1b90afce; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@38060b85; comparator=cascading.tuple.hadoop.util.TupleComparator@14de657f; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:21:46,099 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:21:46,101 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:21:46,101 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:21:46,102 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:21:46,102 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:21:46,099 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:21:46,104 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:21:46,104 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:21:46,104 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 15:21:46,105 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:46,105 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E3D634B224BA49CDA9B6F031ABA527A2, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:21:46,109 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784905721_0001_1_01_000000_0
2016-01-14 15:21:46,109 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E3D634B224BA49CDA9B6F031ABA527A2, mem on close (mb), free: 153, total: 404, max: 672
2016-01-14 15:21:46,110 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:21:46,110 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:21:46,110 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-01-14 15:21:46,110 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-01-14 15:21:46,115 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:21:46,124 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:21:46,125 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:21:46 Completed running task attempt: attempt_1452784905721_0001_1_01_000000_0
2016-01-14 15:21:46,125 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:21:46,126 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:21:46,126 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:21:46,127 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-01-14 15:21:46,127 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:46,127 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F21901D11B07417BA7F4F6C02C4B80F1, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:21:46,125 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784905721_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:46,132 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784905721_0001_1_00_000000_0
2016-01-14 15:21:46,132 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F21901D11B07417BA7F4F6C02C4B80F1, mem on close (mb), free: 152, total: 404, max: 672
2016-01-14 15:21:46,133 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:21:46,133 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:21:46,133 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-01-14 15:21:46,133 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-01-14 15:21:46,134 WARN task.TaskReporter (TaskReporter.java:call(199)) - Exiting TaskReporter thread with pending queue size=2
2016-01-14 15:21:46,135 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784905721_0001_1_01, eventCount=1
2016-01-14 15:21:46,135 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=423624704, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452784906105, Process_Duration=4, Process_End_Time=1452784906109, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-01-14 15:21:46,135 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:46,135 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:46,136 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E3D634B224BA49CDA9B6F031ABA527A2, taskAttemptId=attempt_1452784905721_0001_1_01_000000_0, startTime=1452784905993, finishTime=1452784906135, timeTaken=142, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=423624704, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784906105, Process_Duration=4, Process_End_Time=1452784906109, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 15:21:46,136 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784905721_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:46,136 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:46,136 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784905721_0001_1_01_000000_0
2016-01-14 15:21:46,136 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:46,138 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:46,138 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:TASK_FINISHED]: vertexName=E3D634B224BA49CDA9B6F031ABA527A2, taskId=task_1452784905721_0001_1_01_000000, startTime=1452784905993, finishTime=1452784906137, timeTaken=144, status=SUCCEEDED, successfulAttemptID=attempt_1452784905721_0001_1_01_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=423624704, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784906105, Process_Duration=4, Process_End_Time=1452784906109, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 15:21:46,138 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784905721_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:46,138 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784905721_0001_1_01 [E3D634B224BA49CDA9B6F031ABA527A2] : 1
2016-01-14 15:21:46,139 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784905721_0001_1_01 [E3D634B224BA49CDA9B6F031ABA527A2]
2016-01-14 15:21:46,142 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:VERTEX_FINISHED]: vertexName=E3D634B224BA49CDA9B6F031ABA527A2, vertexId=vertex_1452784905721_0001_1_01, initRequestedTime=1452784905924, initedTime=1452784905961, startRequestedTime=1452784905964, startedTime=1452784905964, finishTime=1452784906139, timeTaken=175, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=423624704, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784906105, Process_Duration=4, Process_End_Time=1452784906109, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1452784905993, firstTasksToStart=[ task_1452784905721_0001_1_01_000000 ], lastTaskFinishTime=1452784906135, lastTasksToFinish=[ task_1452784905721_0001_1_01_000000 ], minTaskDuration=142, maxTaskDuration=142, avgTaskDuration=142.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784905721_0001_1_01_000000 ], longestDurationTasks=[ task_1452784905721_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:46,142 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:46,143 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784905721_0001_1_01 [E3D634B224BA49CDA9B6F031ABA527A2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:46,144 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784905721_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:46,144 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452784905721_0001_1_02 [EBCB27F1E25243EAB2D9E226D90BE526] attempt: attempt_1452784905721_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:21:46,145 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784905721_0001_00_000002
2016-01-14 15:21:46,145 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784905721_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:21:46,145 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784905721_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:46,145 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784905721_0001_00_000002, stoppedTime=1452784906145, exitStatus=0
2016-01-14 15:21:46,145 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:46,146 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784905721_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:46,146 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784905721_0001_1_01 [E3D634B224BA49CDA9B6F031ABA527A2] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 15:21:46,146 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:21:46,146 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452784905721_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 15:21:46,147 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784905721_0001_00_000002, Exiting
2016-01-14 15:21:46,147 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784905721_0001_00_000002
2016-01-14 15:21:46,147 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784905721_0001_00_000002
2016-01-14 15:21:46,149 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784905721_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:46,149 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784905721_0001_00_000002 completed successfully
2016-01-14 15:21:46,151 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784905721_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:21:46,151 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784905721_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:21:46 Completed running task attempt: attempt_1452784905721_0001_1_00_000000_0
2016-01-14 15:21:46,154 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784905721_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:46,155 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=8002, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=423624704, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452784906127, Process_Duration=5, Process_End_Time=1452784906132, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-01-14 15:21:46,155 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:46,156 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:46,156 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784905721_0001_1_00, eventCount=1
2016-01-14 15:21:46,157 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784905721_0001_00_000001
2016-01-14 15:21:46,158 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:46,159 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F21901D11B07417BA7F4F6C02C4B80F1, taskAttemptId=attempt_1452784905721_0001_1_00_000000_0, startTime=1452784905990, finishTime=1452784906157, timeTaken=167, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=8002, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=423624704, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784906127, Process_Duration=5, Process_End_Time=1452784906132, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 15:21:46,159 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784905721_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:46,160 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:46,160 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784905721_0001_1_00_000000_0
2016-01-14 15:21:46,161 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:46,163 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:TASK_FINISHED]: vertexName=F21901D11B07417BA7F4F6C02C4B80F1, taskId=task_1452784905721_0001_1_00_000000, startTime=1452784905990, finishTime=1452784906161, timeTaken=171, status=SUCCEEDED, successfulAttemptID=attempt_1452784905721_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=8002, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=423624704, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784906127, Process_Duration=5, Process_End_Time=1452784906132, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 15:21:46,163 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784905721_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:46,164 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784905721_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:46,164 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:46,165 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784905721_0001_00_000001, stoppedTime=1452784906164, exitStatus=0
2016-01-14 15:21:46,165 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784905721_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:46,166 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784905721_0001_1_00 [F21901D11B07417BA7F4F6C02C4B80F1] : 1
2016-01-14 15:21:46,166 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784905721_0001_1_00 [F21901D11B07417BA7F4F6C02C4B80F1]
2016-01-14 15:21:46,167 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:46,168 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784905721_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:21:46,168 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784905721_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:46,168 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784905721_0001_00_000001, Exiting
2016-01-14 15:21:46,168 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784905721_0001_00_000001
2016-01-14 15:21:46,168 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784905721_0001_00_000001
2016-01-14 15:21:46,170 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784905721_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:46,170 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784905721_0001_00_000001 completed successfully
2016-01-14 15:21:46,170 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:VERTEX_FINISHED]: vertexName=F21901D11B07417BA7F4F6C02C4B80F1, vertexId=vertex_1452784905721_0001_1_00, initRequestedTime=1452784905922, initedTime=1452784905955, startRequestedTime=1452784905958, startedTime=1452784905958, finishTime=1452784906167, timeTaken=209, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=8002, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=423624704, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784906127, Process_Duration=5, Process_End_Time=1452784906132, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1452784905990, firstTasksToStart=[ task_1452784905721_0001_1_00_000000 ], lastTaskFinishTime=1452784906157, lastTasksToFinish=[ task_1452784905721_0001_1_00_000000 ], minTaskDuration=167, maxTaskDuration=167, avgTaskDuration=167.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784905721_0001_1_00_000000 ], longestDurationTasks=[ task_1452784905721_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:46,170 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784905721_0001_1_00 [F21901D11B07417BA7F4F6C02C4B80F1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:46,172 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452784905721_0001_00_000001
2016-01-14 15:21:46,172 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452784905721_0001_1_02 [EBCB27F1E25243EAB2D9E226D90BE526] attempt: attempt_1452784905721_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:21:46,173 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: EBCB27F1E25243EAB2D9E226D90BE526
2016-01-14 15:21:46,174 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784905721_0001_1_02 [EBCB27F1E25243EAB2D9E226D90BE526]
2016-01-14 15:21:46,177 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784905721_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:21:46,178 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784905721_0001_1_00 [F21901D11B07417BA7F4F6C02C4B80F1] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 15:21:46,179 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:TASK_STARTED]: vertexName=EBCB27F1E25243EAB2D9E226D90BE526, taskId=task_1452784905721_0001_1_02_000000, scheduledTime=1452784906179, launchTime=1452784906179
2016-01-14 15:21:46,179 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:46,180 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784905721_0001_1_02_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:46,181 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784905721_0001_1_02 [EBCB27F1E25243EAB2D9E226D90BE526]
2016-01-14 15:21:46,182 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A92072F8C7B04072847C4ED977E8C48B/B9BEEE83742C461BAEBD9AA0318B3850] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: EBCB27F1E25243EAB2D9E226D90BE526, VertexParallelism: 1, TaskAttemptID:attempt_1452784905721_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E3D634B224BA49CDA9B6F031ABA527A2, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=F21901D11B07417BA7F4F6C02C4B80F1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=361A3BFB47874712A1226A1DD07D1243, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:46,183 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784905721_0001_1_02_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:46,184 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:46,184 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784905721_0001_00_000003
2016-01-14 15:21:46,185 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784905721_0001_00_000003 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:46,186 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784905721_0001_00_000003
2016-01-14 15:21:46,187 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 15:21:46,192 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784905721_0001_00_000003 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:46,192 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:46,193 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784905721_0001_00_000003, launchTime=1452784906192
2016-01-14 15:21:46,193 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784905721_0001_00_000003
2016-01-14 15:21:46,194 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784905721_0001_1_02_000000_0] to container: [container_1452784905721_0001_00_000003]
2016-01-14 15:21:46,194 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784905721_0001_00_000003 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:46,195 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784905721_0001_1_02_000000_0] started. Is using containerId: [container_1452784905721_0001_00_000003] on NM: [127.0.0.1:0]
2016-01-14 15:21:46,195 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EBCB27F1E25243EAB2D9E226D90BE526, taskAttemptId=attempt_1452784905721_0001_1_02_000000_0, startTime=1452784906195, containerId=container_1452784905721_0001_00_000003, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784905721_0001_00_000003/teamcity, completedLogs=
2016-01-14 15:21:46,195 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784905721_0001_1_02_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:46,195 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784905721_0001_1_02_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:46,196 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:46,196 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784905721_0001_00_000003 given task: attempt_1452784905721_0001_1_02_000000_0
2016-01-14 15:21:46,197 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784905721_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784905721_0001_1_02_000000_0
2016-01-14 15:21:46 Starting to run new task attempt: attempt_1452784905721_0001_1_02_000000_0
2016-01-14 15:21:46,198 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784905721_0001_1_02_000000_0
2016-01-14 15:21:46,198 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:46,198 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:46,198 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:46,198 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:46,198 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:46,198 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:46,198 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A92072F8C7B04072847C4ED977E8C48B/B9BEEE83742C461BAEBD9AA0318B3850] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: EBCB27F1E25243EAB2D9E226D90BE526, VertexParallelism: 1, TaskAttemptID:attempt_1452784905721_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E3D634B224BA49CDA9B6F031ABA527A2, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=F21901D11B07417BA7F4F6C02C4B80F1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=361A3BFB47874712A1226A1DD07D1243, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:46,199 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:46,205 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784905721_0001_1_02_000000_0
2016-01-14 15:21:46,206 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:46,206 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:46,207 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E3D634B224BA49CDA9B6F031ABA527A2, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:21:46,208 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:46,208 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E3D634B224BA49CDA9B6F031ABA527A2
2016-01-14 15:21:46,209 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F21901D11B07417BA7F4F6C02C4B80F1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:21:46,210 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:46,210 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:46,210 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:21:46,211 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:21:46,212 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: E3D634B224BA49CDA9B6F031ABA527A2
2016-01-14 15:21:46,212 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E3D634B224BA49CDA9B6F031ABA527A2
2016-01-14 15:21:46,212 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F21901D11B07417BA7F4F6C02C4B80F1
2016-01-14 15:21:46,215 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:46,214 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=361A3BFB47874712A1226A1DD07D1243, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:21:46,212 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784905721_0001_1_02_000000_0 sent events: (0-2)
2016-01-14 15:21:46,218 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:46,218 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 361A3BFB47874712A1226A1DD07D1243
2016-01-14 15:21:46,219 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:21:46,219 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 361A3BFB47874712A1226A1DD07D1243
2016-01-14 15:21:46,222 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:21:46,222 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:21:46,223 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: F21901D11B07417BA7F4F6C02C4B80F1
2016-01-14 15:21:46,224 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F21901D11B07417BA7F4F6C02C4B80F1
2016-01-14 15:21:46,232 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EBCB27F1E25243EAB2D9E226D90BE526, ordinal: 2
2016-01-14 15:21:46,232 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EBCB27F1E25243EAB2D9E226D90BE526, mem on start (mb), free: 146, total: 404, max: 672
2016-01-14 15:21:46,232 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:46,232 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 3 initializers to finish
2016-01-14 15:21:46,232 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:46,232 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:46,242 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:21:46,242 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 361A3BFB47874712A1226A1DD07D1243
2016-01-14 15:21:46,242 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:46,243 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:46,243 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:21:46,243 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:21:46,244 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:46,244 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:21:46,244 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345
2016-01-14 15:21:46,245 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 1268357504, TotalRequestedScaled: 6.0881160192E8, TotalJVMHeap: 704643072, TotalAvailable: 461541212, TotalRequested/TotalJVMHeap:1.80
2016-01-14 15:21:46,245 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E3D634B224BA49CDA9B6F031ABA527A2, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-01-14 15:21:46,245 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 361A3BFB47874712A1226A1DD07D1243, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:21:46,245 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F21901D11B07417BA7F4F6C02C4B80F1, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-01-14 15:21:46,246 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:46,246 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E3D634B224BA49CDA9B6F031ABA527A2 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:46,246 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E3D634B224BA49CDA9B6F031ABA527A2
2016-01-14 15:21:46,246 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F21901D11B07417BA7F4F6C02C4B80F1 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:46,251 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:21:46,251 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 2
2016-01-14 15:21:46,251 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:21:46,251 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E3D634B224BA49CDA9B6F031ABA527A2 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:21:46,252 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-01-14 15:21:46,252 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:21:46,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F21901D11B07417BA7F4F6C02C4B80F1
2016-01-14 15:21:46,252 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:21:46,251 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 2 IOs to start
2016-01-14 15:21:46,256 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E3D634B224BA49CDA9B6F031ABA527A2
2016-01-14 15:21:46,256 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:21:46,257 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:21:46,258 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F21901D11B07417BA7F4F6C02C4B80F1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:21:46,258 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-01-14 15:21:46,258 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:21:46,258 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:21:46,259 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:46,260 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F21901D11B07417BA7F4F6C02C4B80F1
2016-01-14 15:21:46,260 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:46,260 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784905721_0001_1_02_000000_0
2016-01-14 15:21:46,262 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452784905721_0001_1_01_000000_0_10027, runDuration: 77841, ]
2016-01-14 15:21:46,264 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452784905721_0001_1_00_000000_0_10026, runDuration: 116002, ]
2016-01-14 15:21:46,266 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452784905721_0001_1_01_000000_0_10027] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1452784906265, TimeTaken=1, Rate=0.09 MB/s
2016-01-14 15:21:46,266 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E3D634B224BA49CDA9B6F031ABA527A2
2016-01-14 15:21:46,267 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]]
2016-01-14 15:21:46,267 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:21:46,268 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] streamed: true, id: 300594D316AC4B119BDC92ADD1A3FAEC
2016-01-14 15:21:46,268 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown"], id: 361A3BFB47874712A1226A1DD07D1243
2016-01-14 15:21:46,268 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown"]
2016-01-14 15:21:46,268 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] 300594D316AC4B119BDC92ADD1A3FAEC, for 2 inputs
2016-01-14 15:21:46,269 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] 300594D316AC4B119BDC92ADD1A3FAEC, for 2 inputs
2016-01-14 15:21:46,268 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [E3D634B224BA49CDA9B6F031ABA527A2] #4 in 5ms
2016-01-14 15:21:46,272 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E3D634B224BA49CDA9B6F031ABA527A2] #4, status:false, isInterrupted:false
2016-01-14 15:21:46,273 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:21:46,273 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-01-14 15:21:46,273 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:21:46,273 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:21:46,274 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-01-14 15:21:46,274 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E3D634B224BA49CDA9B6F031ABA527A2
2016-01-14 15:21:46,274 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:21:46,275 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452784905721_0001_1_00_000000_0_10026] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1452784906275, TimeTaken=3, Rate=0.03 MB/s
2016-01-14 15:21:46,275 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F21901D11B07417BA7F4F6C02C4B80F1
2016-01-14 15:21:46,276 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:21:46,276 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [F21901D11B07417BA7F4F6C02C4B80F1] #5 in 5ms
2016-01-14 15:21:46,276 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F21901D11B07417BA7F4F6C02C4B80F1] #5, status:false, isInterrupted:false
2016-01-14 15:21:46,278 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(89)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2016-01-14 15:21:46,278 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(94)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2016-01-14 15:21:46,280 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:21:46,280 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-01-14 15:21:46,280 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:21:46,280 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:21:46,281 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-01-14 15:21:46,281 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F21901D11B07417BA7F4F6C02C4B80F1
2016-01-14 15:21:46,281 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:21:46,287 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EBCB27F1E25243EAB2D9E226D90BE526, all 2 inputs ready in: 00:00:00.000
2016-01-14 15:21:46,288 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@4d863446; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@25d2076d
2016-01-14 15:21:46,289 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@7fe59880; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3a7f3d19
2016-01-14 15:21:46,295 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:21:46,296 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:21:46,296 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452784905721_0001_1_02_000000_0
2016-01-14 15:21:46,297 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452784905721_0001_1_02_000000_0 given a go for committing the task output.
2016-01-14 15:21:46,298 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784905721_0001_1_02_000000_0
2016-01-14 15:21:46,298 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EBCB27F1E25243EAB2D9E226D90BE526, mem on close (mb), free: 141, total: 404, max: 672
2016-01-14 15:21:46,298 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E3D634B224BA49CDA9B6F031ABA527A2
2016-01-14 15:21:46,299 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:21:46,299 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:21:46,299 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:21:46,300 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:21:46,300 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:21:46,300 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F21901D11B07417BA7F4F6C02C4B80F1
2016-01-14 15:21:46,301 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:21:46,301 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:21:46,302 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:21:46,302 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:21:46,302 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:21:46 Completed running task attempt: attempt_1452784905721_0001_1_02_000000_0
2016-01-14 15:21:46,303 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784905721_0001_1_02_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:46,305 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=423624704, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=23, FIRST_EVENT_RECEIVED=12, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452784906287, Process_Duration=11, Process_End_Time=1452784906298, Read_Duration=0, Tuples_Read=10, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-01-14 15:21:46,305 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EBCB27F1E25243EAB2D9E226D90BE526, taskAttemptId=attempt_1452784905721_0001_1_02_000000_0, startTime=1452784906195, finishTime=1452784906305, timeTaken=110, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=423624704, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=23, FIRST_EVENT_RECEIVED=12, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784906287, Process_Duration=11, Process_End_Time=1452784906298, Read_Duration=0, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-01-14 15:21:46,306 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784905721_0001_1_02_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:46,306 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:46,306 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784905721_0001_1_02_000000_0
2016-01-14 15:21:46,306 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:46,306 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:46,307 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:46,307 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784905721_0001_00_000003
2016-01-14 15:21:46,307 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:46,307 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:TASK_FINISHED]: vertexName=EBCB27F1E25243EAB2D9E226D90BE526, taskId=task_1452784905721_0001_1_02_000000, startTime=1452784906195, finishTime=1452784906306, timeTaken=111, status=SUCCEEDED, successfulAttemptID=attempt_1452784905721_0001_1_02_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=423624704, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=23, FIRST_EVENT_RECEIVED=12, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784906287, Process_Duration=11, Process_End_Time=1452784906298, Read_Duration=0, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-01-14 15:21:46,307 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784905721_0001_1_02_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:46,308 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784905721_0001_1_02 [EBCB27F1E25243EAB2D9E226D90BE526] : 1
2016-01-14 15:21:46,309 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784905721_0001_1_02 [EBCB27F1E25243EAB2D9E226D90BE526]
2016-01-14 15:21:46,310 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:46,312 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:VERTEX_FINISHED]: vertexName=EBCB27F1E25243EAB2D9E226D90BE526, vertexId=vertex_1452784905721_0001_1_02, initRequestedTime=1452784905936, initedTime=1452784905948, startRequestedTime=1452784905970, startedTime=1452784905970, finishTime=1452784906309, timeTaken=339, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=423624704, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=23, FIRST_EVENT_RECEIVED=12, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784906287, Process_Duration=11, Process_End_Time=1452784906298, Read_Duration=0, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1452784906195, firstTasksToStart=[ task_1452784905721_0001_1_02_000000 ], lastTaskFinishTime=1452784906305, lastTasksToFinish=[ task_1452784905721_0001_1_02_000000 ], minTaskDuration=110, maxTaskDuration=110, avgTaskDuration=110.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784905721_0001_1_02_000000 ], longestDurationTasks=[ task_1452784905721_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:46,312 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784905721_0001_1_02 [EBCB27F1E25243EAB2D9E226D90BE526] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:46,313 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784905721_0001_00_000003 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:46,313 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784905721_0001_00_000003, stoppedTime=1452784906313, exitStatus=0
2016-01-14 15:21:46,313 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:46,313 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784905721_0001_00_000003 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:46,314 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784905721_0001_1_02 [EBCB27F1E25243EAB2D9E226D90BE526] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 15:21:46,314 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452784905721_0001_1
2016-01-14 15:21:46,315 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452784905721_0001_1
2016-01-14 15:21:46,315 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452784905721_0001_1_00 [F21901D11B07417BA7F4F6C02C4B80F1]
2016-01-14 15:21:46,315 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452784905721_0001_1_01 [E3D634B224BA49CDA9B6F031ABA527A2]
2016-01-14 15:21:46,315 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 361A3BFB47874712A1226A1DD07D1243 for vertex: vertex_1452784905721_0001_1_02 [EBCB27F1E25243EAB2D9E226D90BE526]
2016-01-14 15:21:46,318 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784905721_0001_00_000003 is valid, but no longer registered, and will be killed
2016-01-14 15:21:46,318 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784905721_0001_00_000003: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:46,319 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784905721_0001_00_000003, Exiting
2016-01-14 15:21:46,319 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784905721_0001_00_000003
2016-01-14 15:21:46,319 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784905721_0001_00_000003
2016-01-14 15:21:46,320 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784905721_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:46,320 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784905721_0001_00_000003 completed successfully
2016-01-14 15:21:46,331 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452784905721_0001_1, queueSize=0
2016-01-14 15:21:46,332 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:21:46,332 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784905721_0001_1][Event:DAG_FINISHED]: dagId=dag_1452784905721_0001_1, startTime=1452784905920, finishTime=1452784906327, timeTaken=407, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=2, File System Counters, FILE_BYTES_READ=440, FILE_BYTES_WRITTEN=8181, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1270874112, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=15, OUTPUT_BYTES=150, OUTPUT_BYTES_WITH_OVERHEAD=182, OUTPUT_BYTES_PHYSICAL=190, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=23, FIRST_EVENT_RECEIVED=12, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4358354718519, Process_Duration=20, Process_End_Time=4358354718539, Read_Duration=6, Tuples_Read=20, Tuples_Written=15, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=5
2016-01-14 15:21:46,332 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452784905721_0001_1 finished with state: SUCCEEDED
2016-01-14 15:21:46,332 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784905721_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:21:46,333 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452784905721_0001_00_000003
2016-01-14 15:21:46,333 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784905721_0001_00_000003 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:21:46 Completed Dag: dag_1452784905721_0001_1
2016-01-14 15:21:46,334 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452784905721_0001_1, dagState=SUCCEEDED
2016-01-14 15:21:46,334 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784905721_0001_1_post
2016-01-14 15:21:46,334 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:46,335 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:21:46,342 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lexcogrouprelativeunknown, applicationId=application_1452784905721_0001
2016-01-14 15:21:46,344 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:21:46,344 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:21:46,344 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:21:46,344 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:21:46,345 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:21:46,345 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:21:46,346 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:21:46,346 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:21:46,346 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:21:46,346 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:21:46,346 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:21:46,347 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:21:46,348 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:21:46,354 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A92072F8C7B04072847C4ED977E8C48B/B9BEEE83742C461BAEBD9AA0318B3850/.tez/application_1452784905721_0001
2016-01-14 15:21:46,355 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] completed in: 00:00.661
2016-01-14 15:21:46,424 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown
2016-01-14 15:21:46,449 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:21:46,450 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:21:46,499 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.023
2016-01-14 15:21:46,519 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.043
2016-01-14 15:21:46,520 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:21:46,520 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:21:46,521 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:21:46,538 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-01-14 15:21:46,546 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative
2016-01-14 15:21:46,721 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 15:21:46,722 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 15:21:46,722 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative"]
2016-01-14 15:21:46,722 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 15:21:46,722 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 15:21:46,723 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 15:21:46,724 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...leRelative/simplerelative
2016-01-14 15:21:46,724 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:21:46,738 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 15:21:46,743 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:46,743 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:21:46,743 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:21:46,752 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/30A373387E644091890633627F02CC91/EE02E523B24A4EAF96E414D98ED8765B/.tez/application_1452784906743_0001 doesn't exist and is created
2016-01-14 15:21:46,767 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:21:46,767 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:21:46,768 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/30A373387E644091890633627F02CC91/EE02E523B24A4EAF96E414D98ED8765B/.tez/application_1452784906743_0001
2016-01-14 15:21:46,771 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452784906743_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:46,781 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:21:46,782 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:21:46,784 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:21:46,784 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:21:46,784 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/30A373387E644091890633627F02CC91/EE02E523B24A4EAF96E414D98ED8765B/.tez/application_1452784906743_0001/localmode-log-dir
2016-01-14 15:21:46,784 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/30A373387E644091890633627F02CC91/EE02E523B24A4EAF96E414D98ED8765B/.tez/application_1452784906743_0001/localmode-log-dir/history.txt.appattempt_1452784906743_0001_000000, maxErrors=10
2016-01-14 15:21:46,784 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:21:46,785 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452784906743_0001_000000, appSubmitTime=1452784906770, launchTime=1452784906770
2016-01-14 15:21:46,787 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:21:46,789 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:21:46,790 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:21:46,790 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:21:46,794 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:38084
2016-01-14 15:21:46,795 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:21:46,798 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452784906743_0001_000000, startTime=1452784906798
2016-01-14 15:21:46,798 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:21:46,798 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:21:46,868 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:21:46,868 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:21:46,868 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...leRelative/simplerelative, applicationId=application_1452784906743_0001, dagName=[30A373387E644091890633627F02CC91/EE02E523B24A4EAF96E414D98ED8765B] test/(1/1) ...leRelative/simplerelative
2016-01-14 15:21:46,887 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [30A373387E644091890633627F02CC91/EE02E523B24A4EAF96E414D98ED8765B] test/(1/1) ...leRelative/simplerelative
2016-01-14 15:21:46,888 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452784906743_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/30A373387E644091890633627F02CC91/EE02E523B24A4EAF96E414D98ED8765B/.tez/application_1452784906743_0001/localmode-log-dir/dag_1452784906743_0001_1.dot
2016-01-14 15:21:46,889 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/30A373387E644091890633627F02CC91/EE02E523B24A4EAF96E414D98ED8765B/.tez/application_1452784906743_0001/localmode-log-dir/dag_1452784906743_0001_1-tez-dag.pb.txt
2016-01-14 15:21:46,919 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784906743_0001_1
2016-01-14 15:21:46,920 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:46,921 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [30A373387E644091890633627F02CC91/EE02E523B24A4EAF96E414D98ED8765B] test/(1/1) ...leRelative/simplerelative
2016-01-14 15:21:46 Running Dag: dag_1452784906743_0001_1
2016-01-14 15:21:46,941 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:21:46,942 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784906743_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452784906743_0001_1, submitTime=1452784906887
2016-01-14 15:21:46,944 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FABBCB6721DB4E02A781647C4BEE52D7
2016-01-14 15:21:46,944 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:21:46,945 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:21:46,946 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784906743_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452784906743_0001_1, initTime=1452784906943
2016-01-14 15:21:46,947 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784906743_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:21:46,948 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784906743_0001_1][Event:DAG_STARTED]: dagID=dag_1452784906743_0001_1, startTime=1452784906947
2016-01-14 15:21:46,948 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:21:46,948 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:21:46,948 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784906743_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:21:46,949 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C1F68270CFBE452DB44B33FD877540EC : {FDE5E338AA424E998137D10A5C012E13={InputName=FDE5E338AA424E998137D10A5C012E13}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:21:46,949 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: FDE5E338AA424E998137D10A5C012E13, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:21:46,949 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452784906743_0001_1_00 [C1F68270CFBE452DB44B33FD877540EC]
2016-01-14 15:21:46,949 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...leRelative/simplerelative, applicationId=application_1452784906743_0001, dagName=[30A373387E644091890633627F02CC91/EE02E523B24A4EAF96E414D98ED8765B] test/(1/1) ...leRelative/simplerelative
2016-01-14 15:21:46,949 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452784906743_0001_1_00 [C1F68270CFBE452DB44B33FD877540EC]
2016-01-14 15:21:46,949 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452784906743_0001_1_00 [C1F68270CFBE452DB44B33FD877540EC]
2016-01-14 15:21:46,950 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452784906743_0001_1_00 [C1F68270CFBE452DB44B33FD877540EC]. Starting root input initializers: 1
2016-01-14 15:21:46,950 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784906743_0001_1_00 [C1F68270CFBE452DB44B33FD877540EC] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:21:46,950 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452784906743_0001_1_01 [FABBCB6721DB4E02A781647C4BEE52D7]
2016-01-14 15:21:46,953 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:21:46,954 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452784906743_0001_1_01 [FABBCB6721DB4E02A781647C4BEE52D7]
2016-01-14 15:21:46,955 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452784906743_0001_1_01 [FABBCB6721DB4E02A781647C4BEE52D7]
2016-01-14 15:21:46,955 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452784906743_0001_1_01 [FABBCB6721DB4E02A781647C4BEE52D7]
2016-01-14 15:21:46,955 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=1B2FC3735A0347A093E475B299FFA867, vertexId=vertex_1452784906743_0001_1_01 [FABBCB6721DB4E02A781647C4BEE52D7], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:21:46,956 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=1B2FC3735A0347A093E475B299FFA867, vertexId=vertex_1452784906743_0001_1_01 [FABBCB6721DB4E02A781647C4BEE52D7]
2016-01-14 15:21:46,958 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=1B2FC3735A0347A093E475B299FFA867, vertexName=FABBCB6721DB4E02A781647C4BEE52D7, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:21:46,959 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=1B2FC3735A0347A093E475B299FFA867, vertexId=vertex_1452784906743_0001_1_01 [FABBCB6721DB4E02A781647C4BEE52D7]
2016-01-14 15:21:46,959 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: FDE5E338AA424E998137D10A5C012E13 on vertex vertex_1452784906743_0001_1_00 [C1F68270CFBE452DB44B33FD877540EC]
2016-01-14 15:21:46,963 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:46,963 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784906743_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FABBCB6721DB4E02A781647C4BEE52D7, vertexId=vertex_1452784906743_0001_1_01, initRequestedTime=1452784906950, initedTime=1452784906963, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:21:46,964 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784906743_0001_1_01 [FABBCB6721DB4E02A781647C4BEE52D7] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:21:46,964 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:21:46,965 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input FDE5E338AA424E998137D10A5C012E13 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:21:46,965 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:21:46,966 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:21:46,968 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-01-14 15:21:46,973 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: FDE5E338AA424E998137D10A5C012E13 on vertex vertex_1452784906743_0001_1_00 [C1F68270CFBE452DB44B33FD877540EC]
2016-01-14 15:21:46,973 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784906743_0001_1_00 [C1F68270CFBE452DB44B33FD877540EC] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:46,974 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {FDE5E338AA424E998137D10A5C012E13=forAllWorkUnits=true, update=[1]}
2016-01-14 15:21:46,974 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452784906743_0001_1_00 [C1F68270CFBE452DB44B33FD877540EC] parallelism set to 1
2016-01-14 15:21:46,975 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784906743_0001_1_00 [C1F68270CFBE452DB44B33FD877540EC] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:46,977 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784906743_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784906743_0001_1_00, eventCount=1
2016-01-14 15:21:46,978 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784906743_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C1F68270CFBE452DB44B33FD877540EC, vertexId=vertex_1452784906743_0001_1_00, initRequestedTime=1452784906949, initedTime=1452784906977, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:21:46,978 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:46,978 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784906743_0001_1_00 [C1F68270CFBE452DB44B33FD877540EC] with distanceFromRoot: 0
2016-01-14 15:21:46,979 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784906743_0001_1_00 [C1F68270CFBE452DB44B33FD877540EC] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:21:46,980 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C1F68270CFBE452DB44B33FD877540EC
2016-01-14 15:21:46,980 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784906743_0001_1_00 [C1F68270CFBE452DB44B33FD877540EC]
2016-01-14 15:21:46,981 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:46,981 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784906743_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784906743_0001_1_00, startRequestedTime=1452784906979, startedTime=1452784906979
2016-01-14 15:21:46,981 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784906743_0001_1_00 [C1F68270CFBE452DB44B33FD877540EC] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:46,982 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:46,982 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784906743_0001_1][Event:TASK_STARTED]: vertexName=C1F68270CFBE452DB44B33FD877540EC, taskId=task_1452784906743_0001_1_00_000000, scheduledTime=1452784906981, launchTime=1452784906981
2016-01-14 15:21:46,982 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784906743_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:46,982 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452784906743_0001_1_00 for vertex: vertex_1452784906743_0001_1_01 [FABBCB6721DB4E02A781647C4BEE52D7] numStartedSources: 1 numSources: 1
2016-01-14 15:21:46,982 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784906743_0001_1_01 [FABBCB6721DB4E02A781647C4BEE52D7] with distanceFromRoot: 1
2016-01-14 15:21:46,983 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784906743_0001_1_00 [C1F68270CFBE452DB44B33FD877540EC]
2016-01-14 15:21:46,983 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C1F68270CFBE452DB44B33FD877540EC in vertex: FABBCB6721DB4E02A781647C4BEE52D7
2016-01-14 15:21:46,983 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: FABBCB6721DB4E02A781647C4BEE52D7 with 1 source tasks and 1 pending tasks
2016-01-14 15:21:46,983 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:46,984 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784906743_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784906743_0001_1_01, startRequestedTime=1452784906983, startedTime=1452784906983
2016-01-14 15:21:46,984 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784906743_0001_1_01 [FABBCB6721DB4E02A781647C4BEE52D7] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:46,984 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [30A373387E644091890633627F02CC91/EE02E523B24A4EAF96E414D98ED8765B] test/(1/1) ...leRelative/simplerelative, VertexName: C1F68270CFBE452DB44B33FD877540EC, VertexParallelism: 1, TaskAttemptID:attempt_1452784906743_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FDE5E338AA424E998137D10A5C012E13, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FABBCB6721DB4E02A781647C4BEE52D7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:21:46,984 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784906743_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:46,989 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:46,998 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:21:46,998 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:21:46,998 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:21:46,999 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784906743_0001_00_000001
2016-01-14 15:21:46,999 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784906743_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:47,001 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784906743_0001_00_000001
2016-01-14 15:21:47,001 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:21:47,003 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452784906743_0001, with dag id: dag_1452784906743_0001_1
2016-01-14 15:21:47,009 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784906743_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:47,009 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:47,009 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784906743_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784906743_0001_00_000001, launchTime=1452784907009
2016-01-14 15:21:47,018 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784906743_0001_00_000001
2016-01-14 15:21:47,020 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784906743_0001_1_00_000000_0] to container: [container_1452784906743_0001_00_000001]
2016-01-14 15:21:47,020 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784906743_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:47,020 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784906743_0001_00_000001 given task: attempt_1452784906743_0001_1_00_000000_0
2016-01-14 15:21:47,020 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784906743_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784906743_0001_1_00_000000_0
2016-01-14 15:21:47 Starting to run new task attempt: attempt_1452784906743_0001_1_00_000000_0
2016-01-14 15:21:47,021 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784906743_0001_1_00_000000_0
2016-01-14 15:21:47,021 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:47,021 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:47,021 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:47,021 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:47,022 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:47,022 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784906743_0001_1_00_000000_0] started. Is using containerId: [container_1452784906743_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:21:47,022 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:47,022 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [30A373387E644091890633627F02CC91/EE02E523B24A4EAF96E414D98ED8765B] test/(1/1) ...leRelative/simplerelative, VertexName: C1F68270CFBE452DB44B33FD877540EC, VertexParallelism: 1, TaskAttemptID:attempt_1452784906743_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FDE5E338AA424E998137D10A5C012E13, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FABBCB6721DB4E02A781647C4BEE52D7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:21:47,022 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:47,022 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:47,022 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784906743_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C1F68270CFBE452DB44B33FD877540EC, taskAttemptId=attempt_1452784906743_0001_1_00_000000_0, startTime=1452784907020, containerId=container_1452784906743_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784906743_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:21:47,023 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784906743_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:47,027 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784906743_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:47,028 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784906743_0001_1_00_000000_0
2016-01-14 15:21:47,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:47,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:47,028 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784906743_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:21:47,031 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=FDE5E338AA424E998137D10A5C012E13, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:21:47,031 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:47,032 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: FDE5E338AA424E998137D10A5C012E13
2016-01-14 15:21:47,032 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: FDE5E338AA424E998137D10A5C012E13
2016-01-14 15:21:47,032 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:47,038 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:21:47,038 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: FDE5E338AA424E998137D10A5C012E13
2016-01-14 15:21:47,038 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: FDE5E338AA424E998137D10A5C012E13
2016-01-14 15:21:47,039 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:47,048 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FABBCB6721DB4E02A781647C4BEE52D7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:21:47,049 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:47,050 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FABBCB6721DB4E02A781647C4BEE52D7
2016-01-14 15:21:47,052 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:21:47,052 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: FABBCB6721DB4E02A781647C4BEE52D7
2016-01-14 15:21:47,053 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FABBCB6721DB4E02A781647C4BEE52D7
2016-01-14 15:21:47,054 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C1F68270CFBE452DB44B33FD877540EC, ordinal: 0
2016-01-14 15:21:47,054 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C1F68270CFBE452DB44B33FD877540EC, mem on start (mb), free: 380, total: 442, max: 672
2016-01-14 15:21:47,054 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:47,055 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:47,055 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:47,055 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:47,055 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:47,056 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:21:47,056 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:47,057 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:21:47,057 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:21:47,058 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:21:47,058 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, FDE5E338AA424E998137D10A5C012E13, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:21:47,058 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FABBCB6721DB4E02A781647C4BEE52D7, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:21:47,059 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:47,059 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: FDE5E338AA424E998137D10A5C012E13 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:47,059 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: FDE5E338AA424E998137D10A5C012E13
2016-01-14 15:21:47,060 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:47,060 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: FDE5E338AA424E998137D10A5C012E13
2016-01-14 15:21:47,060 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:21:47,061 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:47,061 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:47,062 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784906743_0001_1_00_000000_0
2016-01-14 15:21:47,065 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 15:21:47,066 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: FDE5E338AA424E998137D10A5C012E13
2016-01-14 15:21:47,066 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: 9CFCA806ACE248D4988A3B37EF09F4B2
2016-01-14 15:21:47,066 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] 9CFCA806ACE248D4988A3B37EF09F4B2
2016-01-14 15:21:47,067 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@3b954661; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@453fa8d6; comparator=cascading.tuple.hadoop.util.TupleComparator@56659d1e; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:21:47,069 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:21:47,069 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:21:47,070 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:21:47,073 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:21:47,075 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@369b0b39
2016-01-14 15:21:47,085 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@369b0b39
2016-01-14 15:21:47,086 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:21:47,087 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:21:47,091 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:21:47,091 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:21:47,092 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:21:47,092 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:21:47,093 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:21:47,095 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 15:21:47,095 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:47,095 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C1F68270CFBE452DB44B33FD877540EC, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:21:47,098 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784906743_0001_1_00_000000_0
2016-01-14 15:21:47,099 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C1F68270CFBE452DB44B33FD877540EC, mem on close (mb), free: 278, total: 442, max: 672
2016-01-14 15:21:47,099 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:21:47,099 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:21:47,099 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 210; bufvoid = 104857600
2016-01-14 15:21:47,100 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214360(104857440); length = 37/6553600
2016-01-14 15:21:47,106 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:21:47 Completed running task attempt: attempt_1452784906743_0001_1_00_000000_0
2016-01-14 15:21:47,111 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784906743_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:47,112 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784906743_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784906743_0001_1_00, eventCount=1
2016-01-14 15:21:47,112 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=1272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=463994880, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452784907096, Process_Duration=2, Process_End_Time=1452784907098, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10]]
2016-01-14 15:21:47,113 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784906743_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C1F68270CFBE452DB44B33FD877540EC, taskAttemptId=attempt_1452784906743_0001_1_00_000000_0, startTime=1452784907020, finishTime=1452784907113, timeTaken=93, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=1272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=463994880, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784907096, Process_Duration=2, Process_End_Time=1452784907098, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-01-14 15:21:47,113 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:47,113 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:47,113 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784906743_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:47,114 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:47,115 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784906743_0001_00_000001
2016-01-14 15:21:47,115 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:47,115 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784906743_0001_1_00_000000_0
2016-01-14 15:21:47,116 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:47,116 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784906743_0001_1][Event:TASK_FINISHED]: vertexName=C1F68270CFBE452DB44B33FD877540EC, taskId=task_1452784906743_0001_1_00_000000, startTime=1452784907020, finishTime=1452784907116, timeTaken=96, status=SUCCEEDED, successfulAttemptID=attempt_1452784906743_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=1272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=463994880, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784907096, Process_Duration=2, Process_End_Time=1452784907098, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-01-14 15:21:47,116 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784906743_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:47,117 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784906743_0001_1_00 [C1F68270CFBE452DB44B33FD877540EC] : 1
2016-01-14 15:21:47,117 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784906743_0001_1_00 [C1F68270CFBE452DB44B33FD877540EC]
2016-01-14 15:21:47,121 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:47,122 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784906743_0001_1][Event:VERTEX_FINISHED]: vertexName=C1F68270CFBE452DB44B33FD877540EC, vertexId=vertex_1452784906743_0001_1_00, initRequestedTime=1452784906949, initedTime=1452784906977, startRequestedTime=1452784906979, startedTime=1452784906979, finishTime=1452784907117, timeTaken=138, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=1272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=463994880, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784907096, Process_Duration=2, Process_End_Time=1452784907098, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1452784907020, firstTasksToStart=[ task_1452784906743_0001_1_00_000000 ], lastTaskFinishTime=1452784907113, lastTasksToFinish=[ task_1452784906743_0001_1_00_000000 ], minTaskDuration=93, maxTaskDuration=93, avgTaskDuration=93.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784906743_0001_1_00_000000 ], longestDurationTasks=[ task_1452784906743_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:47,122 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784906743_0001_1_00 [C1F68270CFBE452DB44B33FD877540EC] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:47,124 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784906743_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:47,125 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452784906743_0001_1_01 [FABBCB6721DB4E02A781647C4BEE52D7] attempt: attempt_1452784906743_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:21:47,126 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: FABBCB6721DB4E02A781647C4BEE52D7
2016-01-14 15:21:47,127 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784906743_0001_1_01 [FABBCB6721DB4E02A781647C4BEE52D7]
2016-01-14 15:21:47,128 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784906743_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784906743_0001_00_000001, stoppedTime=1452784907128, exitStatus=0
2016-01-14 15:21:47,128 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:47,129 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784906743_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:47,130 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784906743_0001_1_00 [C1F68270CFBE452DB44B33FD877540EC] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:21:47,131 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:47,131 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784906743_0001_1][Event:TASK_STARTED]: vertexName=FABBCB6721DB4E02A781647C4BEE52D7, taskId=task_1452784906743_0001_1_01_000000, scheduledTime=1452784907131, launchTime=1452784907131
2016-01-14 15:21:47,132 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784906743_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:47,132 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452784906743_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:21:47,132 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784906743_0001_1_01 [FABBCB6721DB4E02A781647C4BEE52D7]
2016-01-14 15:21:47,134 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784906743_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:21:47,134 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [30A373387E644091890633627F02CC91/EE02E523B24A4EAF96E414D98ED8765B] test/(1/1) ...leRelative/simplerelative, VertexName: FABBCB6721DB4E02A781647C4BEE52D7, VertexParallelism: 1, TaskAttemptID:attempt_1452784906743_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C1F68270CFBE452DB44B33FD877540EC, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=1B2FC3735A0347A093E475B299FFA867, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:47,135 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784906743_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:47,136 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:47,137 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784906743_0001_00_000002
2016-01-14 15:21:47,137 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784906743_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:47,140 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784906743_0001_00_000002
2016-01-14 15:21:47,140 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:21:47,142 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:47,142 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784906743_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784906743_0001_00_000002, launchTime=1452784907141
2016-01-14 15:21:47,143 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784906743_0001_00_000002
2016-01-14 15:21:47,144 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784906743_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:47,146 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784906743_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:21:47,146 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784906743_0001_00_000001: 31 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:47,146 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784906743_0001_00_000001, Exiting
2016-01-14 15:21:47,146 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784906743_0001_00_000001
2016-01-14 15:21:47,146 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784906743_0001_00_000001
2016-01-14 15:21:47,149 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784906743_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:47,149 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784906743_0001_00_000001 completed successfully
2016-01-14 15:21:47,150 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784906743_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:21:47,155 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784906743_0001_1_01_000000_0] to container: [container_1452784906743_0001_00_000002]
2016-01-14 15:21:47,155 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784906743_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:47,155 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784906743_0001_00_000002 given task: attempt_1452784906743_0001_1_01_000000_0
2016-01-14 15:21:47,155 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784906743_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784906743_0001_1_01_000000_0
2016-01-14 15:21:47 Starting to run new task attempt: attempt_1452784906743_0001_1_01_000000_0
2016-01-14 15:21:47,156 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784906743_0001_1_01_000000_0
2016-01-14 15:21:47,156 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:47,156 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784906743_0001_1_01_000000_0] started. Is using containerId: [container_1452784906743_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:21:47,156 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784906743_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FABBCB6721DB4E02A781647C4BEE52D7, taskAttemptId=attempt_1452784906743_0001_1_01_000000_0, startTime=1452784907156, containerId=container_1452784906743_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784906743_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:21:47,157 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784906743_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:47,157 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784906743_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:47,157 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:47,156 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:47,158 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:47,158 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:47,158 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:47,158 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:47,158 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [30A373387E644091890633627F02CC91/EE02E523B24A4EAF96E414D98ED8765B] test/(1/1) ...leRelative/simplerelative, VertexName: FABBCB6721DB4E02A781647C4BEE52D7, VertexParallelism: 1, TaskAttemptID:attempt_1452784906743_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C1F68270CFBE452DB44B33FD877540EC, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=1B2FC3735A0347A093E475B299FFA867, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:47,158 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:47,163 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784906743_0001_1_01_000000_0
2016-01-14 15:21:47,163 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:47,163 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:47,165 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C1F68270CFBE452DB44B33FD877540EC, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:21:47,166 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:47,167 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C1F68270CFBE452DB44B33FD877540EC
2016-01-14 15:21:47,169 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:21:47,169 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:21:47,170 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C1F68270CFBE452DB44B33FD877540EC
2016-01-14 15:21:47,171 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784906743_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:21:47,170 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:47,172 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=1B2FC3735A0347A093E475B299FFA867, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:21:47,174 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:47,174 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 1B2FC3735A0347A093E475B299FFA867
2016-01-14 15:21:47,174 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:21:47,174 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 1B2FC3735A0347A093E475B299FFA867
2016-01-14 15:21:47,175 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:47,184 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C1F68270CFBE452DB44B33FD877540EC
2016-01-14 15:21:47,192 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FABBCB6721DB4E02A781647C4BEE52D7, ordinal: 1
2016-01-14 15:21:47,192 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FABBCB6721DB4E02A781647C4BEE52D7, mem on start (mb), free: 273, total: 442, max: 672
2016-01-14 15:21:47,192 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:47,192 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:47,193 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:47,199 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:21:47,199 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 1B2FC3735A0347A093E475B299FFA867
2016-01-14 15:21:47,199 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:47,199 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:47,200 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:21:47,200 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:47,200 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:21:47,200 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:21:47,200 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:21:47,200 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 1B2FC3735A0347A093E475B299FFA867, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:21:47,200 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C1F68270CFBE452DB44B33FD877540EC, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:21:47,201 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:47,201 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C1F68270CFBE452DB44B33FD877540EC being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:47,201 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C1F68270CFBE452DB44B33FD877540EC
2016-01-14 15:21:47,201 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:21:47,201 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:47,208 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:21:47,209 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:21:47,210 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C1F68270CFBE452DB44B33FD877540EC with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:21:47,210 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:21:47,210 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:21:47,210 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:21:47,212 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C1F68270CFBE452DB44B33FD877540EC
2016-01-14 15:21:47,213 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:47,218 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784906743_0001_1_01_000000_0
2016-01-14 15:21:47,220 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452784906743_0001_1_00_000000_0_10034, runDuration: 58845, ]
2016-01-14 15:21:47,225 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-01-14 15:21:47,225 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: 9CFCA806ACE248D4988A3B37EF09F4B2
2016-01-14 15:21:47,227 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative"], id: 1B2FC3735A0347A093E475B299FFA867
2016-01-14 15:21:47,228 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative"]
2016-01-14 15:21:47,228 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] 9CFCA806ACE248D4988A3B37EF09F4B2, for 1 inputs
2016-01-14 15:21:47,230 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452784906743_0001_1_00_000000_0_10034] to DISK_DIRECT, CompressedSize=204, DecompressedSize=200,EndTime=1452784907230, TimeTaken=1, Rate=0.19 MB/s
2016-01-14 15:21:47,230 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C1F68270CFBE452DB44B33FD877540EC
2016-01-14 15:21:47,233 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:21:47,233 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [C1F68270CFBE452DB44B33FD877540EC] #6 in 4ms
2016-01-14 15:21:47,233 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C1F68270CFBE452DB44B33FD877540EC] #6, status:false, isInterrupted:false
2016-01-14 15:21:47,235 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:21:47,235 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 204 bytes from disk
2016-01-14 15:21:47,235 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:21:47,235 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:21:47,236 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 175 bytes
2016-01-14 15:21:47,236 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C1F68270CFBE452DB44B33FD877540EC
2016-01-14 15:21:47,237 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:21:47,238 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FABBCB6721DB4E02A781647C4BEE52D7, all 1 inputs ready in: 00:00:00.009
2016-01-14 15:21:47,239 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@58945603; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@729b0c63
2016-01-14 15:21:47,242 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:21:47,243 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:21:47,244 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452784906743_0001_1_01_000000_0
2016-01-14 15:21:47,245 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452784906743_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:21:47,246 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784906743_0001_1_01_000000_0
2016-01-14 15:21:47,247 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FABBCB6721DB4E02A781647C4BEE52D7, mem on close (mb), free: 270, total: 442, max: 672
2016-01-14 15:21:47,247 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C1F68270CFBE452DB44B33FD877540EC
2016-01-14 15:21:47,250 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:21:47,251 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:21:47,252 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:21:47,252 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:21:47,253 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:21:47 Completed running task attempt: attempt_1452784906743_0001_1_01_000000_0
2016-01-14 15:21:47,261 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784906743_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:47,264 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=660, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=463994880, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=24, MERGE_PHASE_TIME=27, FIRST_EVENT_RECEIVED=12, LAST_EVENT_RECEIVED=12][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452784907238, Process_Duration=8, Process_End_Time=1452784907246, Read_Duration=1, Tuples_Read=10, Tuples_Written=8, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=8]]
2016-01-14 15:21:47,264 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784906743_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FABBCB6721DB4E02A781647C4BEE52D7, taskAttemptId=attempt_1452784906743_0001_1_01_000000_0, startTime=1452784907156, finishTime=1452784907264, timeTaken=108, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=660, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=463994880, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=24, MERGE_PHASE_TIME=27, FIRST_EVENT_RECEIVED=12, LAST_EVENT_RECEIVED=12, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784907238, Process_Duration=8, Process_End_Time=1452784907246, Read_Duration=1, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-01-14 15:21:47,264 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:47,265 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784906743_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:47,267 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:47,267 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784906743_0001_1_01_000000_0
2016-01-14 15:21:47,267 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:47,268 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784906743_0001_1][Event:TASK_FINISHED]: vertexName=FABBCB6721DB4E02A781647C4BEE52D7, taskId=task_1452784906743_0001_1_01_000000, startTime=1452784907156, finishTime=1452784907267, timeTaken=111, status=SUCCEEDED, successfulAttemptID=attempt_1452784906743_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=660, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=463994880, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=24, MERGE_PHASE_TIME=27, FIRST_EVENT_RECEIVED=12, LAST_EVENT_RECEIVED=12, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784907238, Process_Duration=8, Process_End_Time=1452784907246, Read_Duration=1, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-01-14 15:21:47,268 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784906743_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:47,268 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784906743_0001_1_01 [FABBCB6721DB4E02A781647C4BEE52D7] : 1
2016-01-14 15:21:47,268 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784906743_0001_1_01 [FABBCB6721DB4E02A781647C4BEE52D7]
2016-01-14 15:21:47,269 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:47,270 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784906743_0001_1][Event:VERTEX_FINISHED]: vertexName=FABBCB6721DB4E02A781647C4BEE52D7, vertexId=vertex_1452784906743_0001_1_01, initRequestedTime=1452784906950, initedTime=1452784906963, startRequestedTime=1452784906983, startedTime=1452784906983, finishTime=1452784907268, timeTaken=285, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=660, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=463994880, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=24, MERGE_PHASE_TIME=27, FIRST_EVENT_RECEIVED=12, LAST_EVENT_RECEIVED=12, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784907238, Process_Duration=8, Process_End_Time=1452784907246, Read_Duration=1, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8, vertexStats=firstTaskStartTime=1452784907156, firstTasksToStart=[ task_1452784906743_0001_1_01_000000 ], lastTaskFinishTime=1452784907264, lastTasksToFinish=[ task_1452784906743_0001_1_01_000000 ], minTaskDuration=108, maxTaskDuration=108, avgTaskDuration=108.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784906743_0001_1_01_000000 ], longestDurationTasks=[ task_1452784906743_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:47,270 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784906743_0001_1_01 [FABBCB6721DB4E02A781647C4BEE52D7] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:47,270 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784906743_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:47,271 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784906743_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784906743_0001_00_000002, stoppedTime=1452784907270, exitStatus=0
2016-01-14 15:21:47,271 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784906743_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:47,271 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784906743_0001_1_01 [FABBCB6721DB4E02A781647C4BEE52D7] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:21:47,271 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452784906743_0001_1
2016-01-14 15:21:47,265 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:47,276 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:47,271 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:47,277 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784906743_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452784906743_0001_1
2016-01-14 15:21:47,278 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 1B2FC3735A0347A093E475B299FFA867 for vertex: vertex_1452784906743_0001_1_01 [FABBCB6721DB4E02A781647C4BEE52D7]
2016-01-14 15:21:47,285 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784906743_0001_00_000002
2016-01-14 15:21:47,285 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784906743_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:21:47,285 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784906743_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:47,285 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784906743_0001_00_000002, Exiting
2016-01-14 15:21:47,285 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784906743_0001_00_000002
2016-01-14 15:21:47,285 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784906743_0001_00_000002
2016-01-14 15:21:47,287 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784906743_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:47,287 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784906743_0001_00_000002 completed successfully
2016-01-14 15:21:47,292 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452784906743_0001_1_00 [C1F68270CFBE452DB44B33FD877540EC]
2016-01-14 15:21:47,295 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452784906743_0001_1, queueSize=0
2016-01-14 15:21:47,296 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:21:47,297 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784906743_0001_1][Event:DAG_FINISHED]: dagId=dag_1452784906743_0001_1, startTime=1452784906947, finishTime=1452784907292, timeTaken=345, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2442, FILE_BYTES_WRITTEN=1932, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=927989760, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=18, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=24, MERGE_PHASE_TIME=27, FIRST_EVENT_RECEIVED=12, LAST_EVENT_RECEIVED=12, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905569814334, Process_Duration=10, Process_End_Time=2905569814344, Read_Duration=2, Tuples_Read=20, Tuples_Written=18, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=8
2016-01-14 15:21:47,297 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452784906743_0001_1 finished with state: SUCCEEDED
2016-01-14 15:21:47,298 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784906743_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:21:47,298 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452784906743_0001_00_000002
2016-01-14 15:21:47,299 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784906743_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:21:47 Completed Dag: dag_1452784906743_0001_1
2016-01-14 15:21:47,300 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452784906743_0001_1, dagState=SUCCEEDED
2016-01-14 15:21:47,300 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784906743_0001_1_post
2016-01-14 15:21:47,300 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:47,301 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:21:47,307 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...leRelative/simplerelative, applicationId=application_1452784906743_0001
2016-01-14 15:21:47,310 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:21:47,310 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:21:47,311 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:21:47,311 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:21:47,311 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:21:47,311 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:21:47,312 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:21:47,313 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:21:47,313 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:21:47,313 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:21:47,313 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:21:47,314 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:21:47,314 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:21:47,320 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/30A373387E644091890633627F02CC91/EE02E523B24A4EAF96E414D98ED8765B/.tez/application_1452784906743_0001
2016-01-14 15:21:47,321 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:00.600
2016-01-14 15:21:47,372 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative
2016-01-14 15:21:47,538 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:21:47,538 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:21:47,604 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.023
2016-01-14 15:21:47,630 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.050
2016-01-14 15:21:47,630 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:21:47,631 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:21:47,631 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:21:47,647 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt
2016-01-14 15:21:47,662 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult
2016-01-14 15:21:47,722 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 15:21:47,723 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 15:21:47,723 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult"]
2016-01-14 15:21:47,724 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 15:21:47,724 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 15:21:47,725 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 15:21:47,727 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...SimpleResult/simpleresult
2016-01-14 15:21:47,727 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:21:47,739 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 15:21:47,744 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:47,745 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:21:47,745 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:21:47,755 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1637F23089C1497E98A4808CCCB38856/A2388A4D95D84BEEB8C80E446500C541/.tez/application_1452784907744_0001 doesn't exist and is created
2016-01-14 15:21:47,771 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:21:47,771 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:21:47,772 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1637F23089C1497E98A4808CCCB38856/A2388A4D95D84BEEB8C80E446500C541/.tez/application_1452784907744_0001
2016-01-14 15:21:47,776 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452784907744_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:47,787 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:21:47,788 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:21:47,789 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:21:47,790 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:21:47,790 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1637F23089C1497E98A4808CCCB38856/A2388A4D95D84BEEB8C80E446500C541/.tez/application_1452784907744_0001/localmode-log-dir
2016-01-14 15:21:47,790 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1637F23089C1497E98A4808CCCB38856/A2388A4D95D84BEEB8C80E446500C541/.tez/application_1452784907744_0001/localmode-log-dir/history.txt.appattempt_1452784907744_0001_000000, maxErrors=10
2016-01-14 15:21:47,791 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:21:47,791 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452784907744_0001_000000, appSubmitTime=1452784907775, launchTime=1452784907775
2016-01-14 15:21:47,797 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:21:47,799 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:21:47,801 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:46428
2016-01-14 15:21:47,802 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:21:47,802 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:21:47,804 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:21:47,807 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452784907744_0001_000000, startTime=1452784907807
2016-01-14 15:21:47,807 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:21:47,807 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:21:47,871 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:21:47,871 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:21:47,871 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...SimpleResult/simpleresult, applicationId=application_1452784907744_0001, dagName=[1637F23089C1497E98A4808CCCB38856/A2388A4D95D84BEEB8C80E446500C541] test/(1/1) ...SimpleResult/simpleresult
2016-01-14 15:21:47,891 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [1637F23089C1497E98A4808CCCB38856/A2388A4D95D84BEEB8C80E446500C541] test/(1/1) ...SimpleResult/simpleresult
2016-01-14 15:21:47,892 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452784907744_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1637F23089C1497E98A4808CCCB38856/A2388A4D95D84BEEB8C80E446500C541/.tez/application_1452784907744_0001/localmode-log-dir/dag_1452784907744_0001_1.dot
2016-01-14 15:21:47,893 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1637F23089C1497E98A4808CCCB38856/A2388A4D95D84BEEB8C80E446500C541/.tez/application_1452784907744_0001/localmode-log-dir/dag_1452784907744_0001_1-tez-dag.pb.txt
2016-01-14 15:21:47,903 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784907744_0001_1
2016-01-14 15:21:47,903 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:47,903 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [1637F23089C1497E98A4808CCCB38856/A2388A4D95D84BEEB8C80E446500C541] test/(1/1) ...SimpleResult/simpleresult
2016-01-14 15:21:47 Running Dag: dag_1452784907744_0001_1
2016-01-14 15:21:47,915 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784907744_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452784907744_0001_1, submitTime=1452784907891
2016-01-14 15:21:47,915 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:21:47,916 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B5A159C266E44A138123F46B34B7BAEC
2016-01-14 15:21:47,917 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:21:47,918 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784907744_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452784907744_0001_1, initTime=1452784907915
2016-01-14 15:21:47,918 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:21:47,920 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784907744_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:21:47,921 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784907744_0001_1][Event:DAG_STARTED]: dagID=dag_1452784907744_0001_1, startTime=1452784907921
2016-01-14 15:21:47,921 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:21:47,921 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784907744_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:21:47,921 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: AD06D536023444F6B4B83CE16A494FB2 : {214595E09CE04E9DA4D217184A705378={InputName=214595E09CE04E9DA4D217184A705378}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:21:47,921 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:21:47,921 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 214595E09CE04E9DA4D217184A705378, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:21:47,922 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452784907744_0001_1_00 [AD06D536023444F6B4B83CE16A494FB2]
2016-01-14 15:21:47,923 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452784907744_0001_1_00 [AD06D536023444F6B4B83CE16A494FB2]
2016-01-14 15:21:47,923 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452784907744_0001_1_00 [AD06D536023444F6B4B83CE16A494FB2]
2016-01-14 15:21:47,923 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452784907744_0001_1_00 [AD06D536023444F6B4B83CE16A494FB2]. Starting root input initializers: 1
2016-01-14 15:21:47,924 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784907744_0001_1_00 [AD06D536023444F6B4B83CE16A494FB2] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:21:47,924 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 214595E09CE04E9DA4D217184A705378 on vertex vertex_1452784907744_0001_1_00 [AD06D536023444F6B4B83CE16A494FB2]
2016-01-14 15:21:47,924 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452784907744_0001_1_01 [B5A159C266E44A138123F46B34B7BAEC]
2016-01-14 15:21:47,925 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:21:47,926 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 214595E09CE04E9DA4D217184A705378 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:21:47,926 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...SimpleResult/simpleresult, applicationId=application_1452784907744_0001, dagName=[1637F23089C1497E98A4808CCCB38856/A2388A4D95D84BEEB8C80E446500C541] test/(1/1) ...SimpleResult/simpleresult
2016-01-14 15:21:47,926 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:21:47,926 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:21:47,929 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-01-14 15:21:47,929 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 214595E09CE04E9DA4D217184A705378 on vertex vertex_1452784907744_0001_1_00 [AD06D536023444F6B4B83CE16A494FB2]
2016-01-14 15:21:47,929 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:21:47,930 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452784907744_0001_1_01 [B5A159C266E44A138123F46B34B7BAEC]
2016-01-14 15:21:47,931 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452784907744_0001_1_01 [B5A159C266E44A138123F46B34B7BAEC]
2016-01-14 15:21:47,931 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452784907744_0001_1_01 [B5A159C266E44A138123F46B34B7BAEC]
2016-01-14 15:21:47,932 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=C9AA2D022CDB4150814EEDE188577532, vertexId=vertex_1452784907744_0001_1_01 [B5A159C266E44A138123F46B34B7BAEC], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:21:47,933 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=C9AA2D022CDB4150814EEDE188577532, vertexId=vertex_1452784907744_0001_1_01 [B5A159C266E44A138123F46B34B7BAEC]
2016-01-14 15:21:47,935 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=C9AA2D022CDB4150814EEDE188577532, vertexName=B5A159C266E44A138123F46B34B7BAEC, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:21:47,936 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=C9AA2D022CDB4150814EEDE188577532, vertexId=vertex_1452784907744_0001_1_01 [B5A159C266E44A138123F46B34B7BAEC]
2016-01-14 15:21:47,938 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:47,938 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784907744_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B5A159C266E44A138123F46B34B7BAEC, vertexId=vertex_1452784907744_0001_1_01, initRequestedTime=1452784907924, initedTime=1452784907938, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:21:47,939 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784907744_0001_1_01 [B5A159C266E44A138123F46B34B7BAEC] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:21:47,939 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784907744_0001_1_00 [AD06D536023444F6B4B83CE16A494FB2] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:47,940 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {214595E09CE04E9DA4D217184A705378=forAllWorkUnits=true, update=[1]}
2016-01-14 15:21:47,941 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452784907744_0001_1_00 [AD06D536023444F6B4B83CE16A494FB2] parallelism set to 1
2016-01-14 15:21:47,941 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784907744_0001_1_00 [AD06D536023444F6B4B83CE16A494FB2] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:47,942 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784907744_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784907744_0001_1_00, eventCount=1
2016-01-14 15:21:47,943 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:47,944 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784907744_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AD06D536023444F6B4B83CE16A494FB2, vertexId=vertex_1452784907744_0001_1_00, initRequestedTime=1452784907921, initedTime=1452784907943, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:21:47,944 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784907744_0001_1_00 [AD06D536023444F6B4B83CE16A494FB2] with distanceFromRoot: 0
2016-01-14 15:21:47,944 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784907744_0001_1_00 [AD06D536023444F6B4B83CE16A494FB2] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:21:47,945 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in AD06D536023444F6B4B83CE16A494FB2
2016-01-14 15:21:47,945 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784907744_0001_1_00 [AD06D536023444F6B4B83CE16A494FB2]
2016-01-14 15:21:47,947 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:47,947 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784907744_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784907744_0001_1_00, startRequestedTime=1452784907945, startedTime=1452784907945
2016-01-14 15:21:47,948 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784907744_0001_1_00 [AD06D536023444F6B4B83CE16A494FB2] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:47,949 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:47,949 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784907744_0001_1][Event:TASK_STARTED]: vertexName=AD06D536023444F6B4B83CE16A494FB2, taskId=task_1452784907744_0001_1_00_000000, scheduledTime=1452784907949, launchTime=1452784907949
2016-01-14 15:21:47,949 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784907744_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:47,950 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452784907744_0001_1_00 for vertex: vertex_1452784907744_0001_1_01 [B5A159C266E44A138123F46B34B7BAEC] numStartedSources: 1 numSources: 1
2016-01-14 15:21:47,950 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784907744_0001_1_01 [B5A159C266E44A138123F46B34B7BAEC] with distanceFromRoot: 1
2016-01-14 15:21:47,951 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784907744_0001_1_00 [AD06D536023444F6B4B83CE16A494FB2]
2016-01-14 15:21:47,952 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: AD06D536023444F6B4B83CE16A494FB2 in vertex: B5A159C266E44A138123F46B34B7BAEC
2016-01-14 15:21:47,952 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B5A159C266E44A138123F46B34B7BAEC with 1 source tasks and 1 pending tasks
2016-01-14 15:21:47,953 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:47,953 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784907744_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784907744_0001_1_01, startRequestedTime=1452784907951, startedTime=1452784907951
2016-01-14 15:21:47,954 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784907744_0001_1_01 [B5A159C266E44A138123F46B34B7BAEC] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:47,955 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1637F23089C1497E98A4808CCCB38856/A2388A4D95D84BEEB8C80E446500C541] test/(1/1) ...SimpleResult/simpleresult, VertexName: AD06D536023444F6B4B83CE16A494FB2, VertexParallelism: 1, TaskAttemptID:attempt_1452784907744_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=214595E09CE04E9DA4D217184A705378, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B5A159C266E44A138123F46B34B7BAEC, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:21:47,955 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784907744_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:47,956 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:47,959 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452784907744_0001, with dag id: dag_1452784907744_0001_1
2016-01-14 15:21:47,960 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:21:47,961 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:21:47,961 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:21:47,962 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784907744_0001_00_000001
2016-01-14 15:21:47,962 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784907744_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:47,965 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784907744_0001_00_000001
2016-01-14 15:21:47,965 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:21:47,969 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784907744_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:47,969 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:47,970 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784907744_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784907744_0001_00_000001, launchTime=1452784907969
2016-01-14 15:21:47,972 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784907744_0001_00_000001
2016-01-14 15:21:47,973 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784907744_0001_1_00_000000_0] to container: [container_1452784907744_0001_00_000001]
2016-01-14 15:21:47,973 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784907744_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:47,974 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784907744_0001_00_000001 given task: attempt_1452784907744_0001_1_00_000000_0
2016-01-14 15:21:47,974 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784907744_0001_1_00_000000_0] started. Is using containerId: [container_1452784907744_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:21:47,975 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784907744_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AD06D536023444F6B4B83CE16A494FB2, taskAttemptId=attempt_1452784907744_0001_1_00_000000_0, startTime=1452784907974, containerId=container_1452784907744_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784907744_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:21:47,975 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784907744_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:47,975 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784907744_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:47,974 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784907744_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784907744_0001_1_00_000000_0
2016-01-14 15:21:47 Starting to run new task attempt: attempt_1452784907744_0001_1_00_000000_0
2016-01-14 15:21:47,976 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784907744_0001_1_00_000000_0
2016-01-14 15:21:47,975 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:47,976 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:47,977 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:47,977 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:47,977 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:47,977 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:47,977 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:47,978 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1637F23089C1497E98A4808CCCB38856/A2388A4D95D84BEEB8C80E446500C541] test/(1/1) ...SimpleResult/simpleresult, VertexName: AD06D536023444F6B4B83CE16A494FB2, VertexParallelism: 1, TaskAttemptID:attempt_1452784907744_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=214595E09CE04E9DA4D217184A705378, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B5A159C266E44A138123F46B34B7BAEC, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:21:47,978 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:47,981 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784907744_0001_1_00_000000_0
2016-01-14 15:21:47,982 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:47,982 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:47,993 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784907744_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:21:47,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:47,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=214595E09CE04E9DA4D217184A705378, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:21:47,996 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:47,996 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 214595E09CE04E9DA4D217184A705378
2016-01-14 15:21:48,002 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 214595E09CE04E9DA4D217184A705378
2016-01-14 15:21:47,997 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B5A159C266E44A138123F46B34B7BAEC, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:21:47,997 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:48,012 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:48,013 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B5A159C266E44A138123F46B34B7BAEC
2016-01-14 15:21:48,013 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:21:48,019 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 214595E09CE04E9DA4D217184A705378
2016-01-14 15:21:48,019 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 214595E09CE04E9DA4D217184A705378
2016-01-14 15:21:48,022 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:21:48,023 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B5A159C266E44A138123F46B34B7BAEC
2016-01-14 15:21:48,023 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B5A159C266E44A138123F46B34B7BAEC
2016-01-14 15:21:48,024 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AD06D536023444F6B4B83CE16A494FB2, ordinal: 0
2016-01-14 15:21:48,024 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AD06D536023444F6B4B83CE16A494FB2, mem on start (mb), free: 262, total: 442, max: 672
2016-01-14 15:21:48,024 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:48,024 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:48,025 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:48,025 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:48,025 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:48,025 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:21:48,025 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:48,026 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:21:48,026 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:21:48,027 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:21:48,027 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 214595E09CE04E9DA4D217184A705378, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:21:48,027 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B5A159C266E44A138123F46B34B7BAEC, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:21:48,027 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:48,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 214595E09CE04E9DA4D217184A705378
2016-01-14 15:21:48,028 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:48,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 214595E09CE04E9DA4D217184A705378
2016-01-14 15:21:48,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 214595E09CE04E9DA4D217184A705378 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:48,029 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:21:48,029 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:48,029 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:48,030 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784907744_0001_1_00_000000_0
2016-01-14 15:21:48,031 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:21:48,045 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@82df887
2016-01-14 15:21:48,045 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 15:21:48,047 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"] streamed: true, id: 214595E09CE04E9DA4D217184A705378
2016-01-14 15:21:48,047 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: A1D72DCE25094F2A832C173033C52687
2016-01-14 15:21:48,048 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] A1D72DCE25094F2A832C173033C52687
2016-01-14 15:21:48,049 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@82df887
2016-01-14 15:21:48,051 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:21:48,052 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:21:48,051 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@5c5c419a; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@2cd714d5; comparator=cascading.tuple.hadoop.util.TupleComparator@5c98c2f2; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:21:48,054 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:21:48,054 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:21:48,054 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:21:48,067 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:21:48,067 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:21:48,068 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:21:48,068 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:21:48,068 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:21:48,145 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 15:21:48,145 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:48,145 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AD06D536023444F6B4B83CE16A494FB2, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:21:48,149 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784907744_0001_1_00_000000_0
2016-01-14 15:21:48,149 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AD06D536023444F6B4B83CE16A494FB2, mem on close (mb), free: 157, total: 442, max: 672
2016-01-14 15:21:48,149 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:21:48,149 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:21:48,149 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 22; bufvoid = 104857600
2016-01-14 15:21:48,149 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214392(104857568); length = 5/6553600
2016-01-14 15:21:48,152 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:21:48 Completed running task attempt: attempt_1452784907744_0001_1_00_000000_0
2016-01-14 15:21:48,155 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784907744_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:48,156 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784907744_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784907744_0001_1_00, eventCount=1
2016-01-14 15:21:48,157 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784907744_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AD06D536023444F6B4B83CE16A494FB2, taskAttemptId=attempt_1452784907744_0001_1_00_000000_0, startTime=1452784907974, finishTime=1452784908157, timeTaken=183, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=68, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=463994880, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784908145, Process_Duration=4, Process_End_Time=1452784908149, Read_Duration=3, Tuples_Read=5, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 15:21:48,157 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784907744_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:48,157 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784907744_0001_1_00_000000_0
2016-01-14 15:21:48,157 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:48,158 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784907744_0001_1][Event:TASK_FINISHED]: vertexName=AD06D536023444F6B4B83CE16A494FB2, taskId=task_1452784907744_0001_1_00_000000, startTime=1452784907974, finishTime=1452784908157, timeTaken=183, status=SUCCEEDED, successfulAttemptID=attempt_1452784907744_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=68, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=463994880, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784908145, Process_Duration=4, Process_End_Time=1452784908149, Read_Duration=3, Tuples_Read=5, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 15:21:48,158 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784907744_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:48,158 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:48,158 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784907744_0001_1_00 [AD06D536023444F6B4B83CE16A494FB2] : 1
2016-01-14 15:21:48,158 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784907744_0001_1_00 [AD06D536023444F6B4B83CE16A494FB2]
2016-01-14 15:21:48,159 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:48,166 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:48,166 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784907744_0001_1][Event:VERTEX_FINISHED]: vertexName=AD06D536023444F6B4B83CE16A494FB2, vertexId=vertex_1452784907744_0001_1_00, initRequestedTime=1452784907921, initedTime=1452784907943, startRequestedTime=1452784907945, startedTime=1452784907945, finishTime=1452784908158, timeTaken=213, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=68, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=463994880, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784908145, Process_Duration=4, Process_End_Time=1452784908149, Read_Duration=3, Tuples_Read=5, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1452784907974, firstTasksToStart=[ task_1452784907744_0001_1_00_000000 ], lastTaskFinishTime=1452784908157, lastTasksToFinish=[ task_1452784907744_0001_1_00_000000 ], minTaskDuration=183, maxTaskDuration=183, avgTaskDuration=183.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784907744_0001_1_00_000000 ], longestDurationTasks=[ task_1452784907744_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:48,166 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784907744_0001_1_00 [AD06D536023444F6B4B83CE16A494FB2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:48,167 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452784907744_0001_1_01 [B5A159C266E44A138123F46B34B7BAEC] attempt: attempt_1452784907744_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:21:48,167 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B5A159C266E44A138123F46B34B7BAEC
2016-01-14 15:21:48,167 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784907744_0001_1_01 [B5A159C266E44A138123F46B34B7BAEC]
2016-01-14 15:21:48,167 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784907744_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:48,167 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784907744_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784907744_0001_00_000001, stoppedTime=1452784908167, exitStatus=0
2016-01-14 15:21:48,168 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784907744_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:48,168 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784907744_0001_1_00 [AD06D536023444F6B4B83CE16A494FB2] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:21:48,168 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784907744_0001_1][Event:TASK_STARTED]: vertexName=B5A159C266E44A138123F46B34B7BAEC, taskId=task_1452784907744_0001_1_01_000000, scheduledTime=1452784908168, launchTime=1452784908168
2016-01-14 15:21:48,168 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784907744_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:48,169 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784907744_0001_1_01 [B5A159C266E44A138123F46B34B7BAEC]
2016-01-14 15:21:48,169 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1637F23089C1497E98A4808CCCB38856/A2388A4D95D84BEEB8C80E446500C541] test/(1/1) ...SimpleResult/simpleresult, VertexName: B5A159C266E44A138123F46B34B7BAEC, VertexParallelism: 1, TaskAttemptID:attempt_1452784907744_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AD06D536023444F6B4B83CE16A494FB2, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C9AA2D022CDB4150814EEDE188577532, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:48,167 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=68, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=463994880, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452784908145, Process_Duration=4, Process_End_Time=1452784908149, Read_Duration=3, Tuples_Read=5, Tuples_Written=2, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-01-14 15:21:48,169 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784907744_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:48,169 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452784907744_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:21:48,171 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:48,171 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:48,171 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:48,174 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:48,174 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:48,171 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784907744_0001_00_000001
2016-01-14 15:21:48,177 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784907744_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:21:48,178 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784907744_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:48,171 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784907744_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:21:48,179 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784907744_0001_00_000002
2016-01-14 15:21:48,179 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784907744_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:48,180 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784907744_0001_00_000002
2016-01-14 15:21:48,181 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:21:48,181 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784907744_0001_00_000001, Exiting
2016-01-14 15:21:48,182 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784907744_0001_00_000001
2016-01-14 15:21:48,182 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784907744_0001_00_000001
2016-01-14 15:21:48,188 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784907744_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:48,189 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:48,189 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784907744_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784907744_0001_00_000002, launchTime=1452784908189
2016-01-14 15:21:48,195 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784907744_0001_00_000002
2016-01-14 15:21:48,195 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784907744_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:48,196 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784907744_0001_00_000001 completed successfully
2016-01-14 15:21:48,196 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784907744_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:21:48,198 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784907744_0001_1_01_000000_0] to container: [container_1452784907744_0001_00_000002]
2016-01-14 15:21:48,198 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784907744_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:48,199 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784907744_0001_1_01_000000_0] started. Is using containerId: [container_1452784907744_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:21:48,199 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784907744_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B5A159C266E44A138123F46B34B7BAEC, taskAttemptId=attempt_1452784907744_0001_1_01_000000_0, startTime=1452784908199, containerId=container_1452784907744_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784907744_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:21:48,199 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:48,199 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784907744_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:48,204 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784907744_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:48,207 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784907744_0001_00_000002 given task: attempt_1452784907744_0001_1_01_000000_0
2016-01-14 15:21:48,211 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784907744_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784907744_0001_1_01_000000_0
2016-01-14 15:21:48 Starting to run new task attempt: attempt_1452784907744_0001_1_01_000000_0
2016-01-14 15:21:48,212 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784907744_0001_1_01_000000_0
2016-01-14 15:21:48,212 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:48,213 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:48,213 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:48,213 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:48,213 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:48,213 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:48,213 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1637F23089C1497E98A4808CCCB38856/A2388A4D95D84BEEB8C80E446500C541] test/(1/1) ...SimpleResult/simpleresult, VertexName: B5A159C266E44A138123F46B34B7BAEC, VertexParallelism: 1, TaskAttemptID:attempt_1452784907744_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AD06D536023444F6B4B83CE16A494FB2, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C9AA2D022CDB4150814EEDE188577532, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:48,214 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:48,215 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784907744_0001_1_01_000000_0
2016-01-14 15:21:48,217 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:48,217 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:48,217 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:48,217 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=AD06D536023444F6B4B83CE16A494FB2, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:21:48,218 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:48,218 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784907744_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:21:48,219 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C9AA2D022CDB4150814EEDE188577532, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:21:48,219 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:48,220 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C9AA2D022CDB4150814EEDE188577532
2016-01-14 15:21:48,220 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:21:48,220 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: C9AA2D022CDB4150814EEDE188577532
2016-01-14 15:21:48,221 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:48,218 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: AD06D536023444F6B4B83CE16A494FB2
2016-01-14 15:21:48,226 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:21:48,226 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:21:48,226 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: AD06D536023444F6B4B83CE16A494FB2
2016-01-14 15:21:48,229 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: AD06D536023444F6B4B83CE16A494FB2
2016-01-14 15:21:48,236 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B5A159C266E44A138123F46B34B7BAEC, ordinal: 1
2016-01-14 15:21:48,236 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B5A159C266E44A138123F46B34B7BAEC, mem on start (mb), free: 151, total: 442, max: 672
2016-01-14 15:21:48,237 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:48,237 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:48,242 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:21:48,242 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C9AA2D022CDB4150814EEDE188577532
2016-01-14 15:21:48,242 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:48,242 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:48,242 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:48,243 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:21:48,244 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:48,244 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:21:48,244 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:21:48,244 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:21:48,244 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C9AA2D022CDB4150814EEDE188577532, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:21:48,245 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, AD06D536023444F6B4B83CE16A494FB2, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:21:48,245 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:48,245 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: AD06D536023444F6B4B83CE16A494FB2 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:48,245 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: AD06D536023444F6B4B83CE16A494FB2
2016-01-14 15:21:48,246 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:21:48,245 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:21:48,247 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:48,249 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:21:48,249 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: AD06D536023444F6B4B83CE16A494FB2 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:21:48,249 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:21:48,249 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:21:48,250 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:21:48,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: AD06D536023444F6B4B83CE16A494FB2
2016-01-14 15:21:48,251 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:48,252 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784907744_0001_1_01_000000_0
2016-01-14 15:21:48,252 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452784907744_0001_1_00_000000_0_10040, runDuration: 135855, ]
2016-01-14 15:21:48,254 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452784907744_0001_1_00_000000_0_10040] to DISK_DIRECT, CompressedSize=36, DecompressedSize=32,EndTime=1452784908254, TimeTaken=1, Rate=0.03 MB/s
2016-01-14 15:21:48,254 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : AD06D536023444F6B4B83CE16A494FB2
2016-01-14 15:21:48,254 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:21:48,254 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [AD06D536023444F6B4B83CE16A494FB2] #7 in 1ms
2016-01-14 15:21:48,255 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [AD06D536023444F6B4B83CE16A494FB2] #7, status:false, isInterrupted:false
2016-01-14 15:21:48,256 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:21:48,257 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 36 bytes from disk
2016-01-14 15:21:48,257 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:21:48,257 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:21:48,257 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 17 bytes
2016-01-14 15:21:48,257 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : AD06D536023444F6B4B83CE16A494FB2
2016-01-14 15:21:48,260 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:21:48,261 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-01-14 15:21:48,262 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: A1D72DCE25094F2A832C173033C52687
2016-01-14 15:21:48,262 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult"], id: C9AA2D022CDB4150814EEDE188577532
2016-01-14 15:21:48,263 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult"]
2016-01-14 15:21:48,263 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] A1D72DCE25094F2A832C173033C52687, for 1 inputs
2016-01-14 15:21:48,263 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B5A159C266E44A138123F46B34B7BAEC, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:21:48,264 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@e4bd54e; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@14a0fb37
2016-01-14 15:21:48,268 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:21:48,269 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:21:48,269 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452784907744_0001_1_01_000000_0
2016-01-14 15:21:48,270 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452784907744_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:21:48,270 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784907744_0001_1_01_000000_0
2016-01-14 15:21:48,271 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B5A159C266E44A138123F46B34B7BAEC, mem on close (mb), free: 149, total: 442, max: 672
2016-01-14 15:21:48,271 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: AD06D536023444F6B4B83CE16A494FB2
2016-01-14 15:21:48,271 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:21:48,272 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:21:48,272 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:21:48,272 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:21:48,272 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:21:48 Completed running task attempt: attempt_1452784907744_0001_1_01_000000_0
2016-01-14 15:21:48,274 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784907744_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:48,276 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784907744_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B5A159C266E44A138123F46B34B7BAEC, taskAttemptId=attempt_1452784907744_0001_1_01_000000_0, startTime=1452784908199, finishTime=1452784908276, timeTaken=77, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=68, FILE_BYTES_WRITTEN=24, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=463994880, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784908264, Process_Duration=6, Process_End_Time=1452784908270, Read_Duration=0, Tuples_Read=2, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=2
2016-01-14 15:21:48,277 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784907744_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:48,277 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:48,277 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784907744_0001_1_01_000000_0
2016-01-14 15:21:48,278 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:48,278 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784907744_0001_1][Event:TASK_FINISHED]: vertexName=B5A159C266E44A138123F46B34B7BAEC, taskId=task_1452784907744_0001_1_01_000000, startTime=1452784908199, finishTime=1452784908277, timeTaken=78, status=SUCCEEDED, successfulAttemptID=attempt_1452784907744_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=68, FILE_BYTES_WRITTEN=24, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=463994880, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784908264, Process_Duration=6, Process_End_Time=1452784908270, Read_Duration=0, Tuples_Read=2, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=2
2016-01-14 15:21:48,278 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784907744_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:48,278 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784907744_0001_1_01 [B5A159C266E44A138123F46B34B7BAEC] : 1
2016-01-14 15:21:48,278 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784907744_0001_1_01 [B5A159C266E44A138123F46B34B7BAEC]
2016-01-14 15:21:48,279 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:48,280 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:48,281 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784907744_0001_1][Event:VERTEX_FINISHED]: vertexName=B5A159C266E44A138123F46B34B7BAEC, vertexId=vertex_1452784907744_0001_1_01, initRequestedTime=1452784907924, initedTime=1452784907938, startRequestedTime=1452784907951, startedTime=1452784907951, finishTime=1452784908278, timeTaken=327, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=68, FILE_BYTES_WRITTEN=24, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=463994880, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784908264, Process_Duration=6, Process_End_Time=1452784908270, Read_Duration=0, Tuples_Read=2, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=2, vertexStats=firstTaskStartTime=1452784908199, firstTasksToStart=[ task_1452784907744_0001_1_01_000000 ], lastTaskFinishTime=1452784908276, lastTasksToFinish=[ task_1452784907744_0001_1_01_000000 ], minTaskDuration=77, maxTaskDuration=77, avgTaskDuration=77.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784907744_0001_1_01_000000 ], longestDurationTasks=[ task_1452784907744_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:48,281 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784907744_0001_1_01 [B5A159C266E44A138123F46B34B7BAEC] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:48,282 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=68, FILE_BYTES_WRITTEN=24, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=463994880, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452784908264, Process_Duration=6, Process_End_Time=1452784908270, Read_Duration=0, Tuples_Read=2, Tuples_Written=2, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=2]]
2016-01-14 15:21:48,282 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784907744_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:48,282 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:48,283 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:48,283 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784907744_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784907744_0001_00_000002, stoppedTime=1452784908283, exitStatus=0
2016-01-14 15:21:48,283 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784907744_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:48,283 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:48,284 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784907744_0001_00_000002
2016-01-14 15:21:48,284 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784907744_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:21:48,285 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784907744_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:48,284 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784907744_0001_1_01 [B5A159C266E44A138123F46B34B7BAEC] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:21:48,285 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452784907744_0001_1
2016-01-14 15:21:48,286 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784907744_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452784907744_0001_1
2016-01-14 15:21:48,286 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452784907744_0001_1_00 [AD06D536023444F6B4B83CE16A494FB2]
2016-01-14 15:21:48,286 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: C9AA2D022CDB4150814EEDE188577532 for vertex: vertex_1452784907744_0001_1_01 [B5A159C266E44A138123F46B34B7BAEC]
2016-01-14 15:21:48,287 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784907744_0001_00_000002, Exiting
2016-01-14 15:21:48,287 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784907744_0001_00_000002
2016-01-14 15:21:48,287 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784907744_0001_00_000002
2016-01-14 15:21:48,288 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784907744_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:48,288 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784907744_0001_00_000002 completed successfully
2016-01-14 15:21:48,313 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452784907744_0001_1, queueSize=0
2016-01-14 15:21:48,313 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:21:48,315 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784907744_0001_1][Event:DAG_FINISHED]: dagId=dag_1452784907744_0001_1, startTime=1452784907921, finishTime=1452784908311, timeTaken=390, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=130, FILE_BYTES_WRITTEN=92, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=4, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=927989760, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=4, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905569816409, Process_Duration=10, Process_End_Time=2905569816419, Read_Duration=3, Tuples_Read=7, Tuples_Written=4, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=2
2016-01-14 15:21:48,315 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452784907744_0001_1 finished with state: SUCCEEDED
2016-01-14 15:21:48,315 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784907744_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:21:48,315 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452784907744_0001_00_000002
2016-01-14 15:21:48,316 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784907744_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:21:48 Completed Dag: dag_1452784907744_0001_1
2016-01-14 15:21:48,317 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452784907744_0001_1, dagState=SUCCEEDED
2016-01-14 15:21:48,317 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784907744_0001_1_post
2016-01-14 15:21:48,317 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:48,317 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:21:48,318 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:21:48,318 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:21:48,324 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...SimpleResult/simpleresult, applicationId=application_1452784907744_0001
2016-01-14 15:21:48,327 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:21:48,327 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:21:48,327 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:21:48,327 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:21:48,327 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:21:48,328 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:21:48,328 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:21:48,329 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:21:48,329 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:21:48,330 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:21:48,330 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:21:48,331 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:21:48,331 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:21:48,336 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1637F23089C1497E98A4808CCCB38856/A2388A4D95D84BEEB8C80E446500C541/.tez/application_1452784907744_0001
2016-01-14 15:21:48,337 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:00.615
2016-01-14 15:21:48,372 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult
2016-01-14 15:21:48,488 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:21:48,488 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:21:48,525 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-01-14 15:21:48,533 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-01-14 15:21:48,533 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 15:21:48,534 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 15:21:48,535 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:21:48,550 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-01-14 15:21:48,550 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall
2016-01-14 15:21:48,560 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 15:21:48,560 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 15:21:48,561 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall"]
2016-01-14 15:21:48,561 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 15:21:48,561 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 15:21:48,562 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 15:21:48,562 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...m/testFilterAll/filterall
2016-01-14 15:21:48,563 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:21:48,575 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 15:21:48,579 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:48,580 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:21:48,580 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:21:48,588 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0C0F02ECA88248028E1309DD30ED57C3/DBBAC2516DE440508D8F650B5EFC20E9/.tez/application_1452784908580_0001 doesn't exist and is created
2016-01-14 15:21:48,606 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:21:48,606 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:21:48,607 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0C0F02ECA88248028E1309DD30ED57C3/DBBAC2516DE440508D8F650B5EFC20E9/.tez/application_1452784908580_0001
2016-01-14 15:21:48,611 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452784908580_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:48,622 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:21:48,622 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:21:48,624 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:21:48,625 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:21:48,625 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0C0F02ECA88248028E1309DD30ED57C3/DBBAC2516DE440508D8F650B5EFC20E9/.tez/application_1452784908580_0001/localmode-log-dir
2016-01-14 15:21:48,625 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0C0F02ECA88248028E1309DD30ED57C3/DBBAC2516DE440508D8F650B5EFC20E9/.tez/application_1452784908580_0001/localmode-log-dir/history.txt.appattempt_1452784908580_0001_000000, maxErrors=10
2016-01-14 15:21:48,625 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:21:48,625 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452784908580_0001_000000, appSubmitTime=1452784908610, launchTime=1452784908610
2016-01-14 15:21:48,630 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:21:48,631 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:21:48,632 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:21:48,632 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:21:48,642 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:36341
2016-01-14 15:21:48,643 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:21:48,646 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452784908580_0001_000000, startTime=1452784908646
2016-01-14 15:21:48,646 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:21:48,646 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:21:48,707 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:21:48,707 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:21:48,707 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...m/testFilterAll/filterall, applicationId=application_1452784908580_0001, dagName=[0C0F02ECA88248028E1309DD30ED57C3/DBBAC2516DE440508D8F650B5EFC20E9] test/(1/1) ...m/testFilterAll/filterall
2016-01-14 15:21:48,724 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [0C0F02ECA88248028E1309DD30ED57C3/DBBAC2516DE440508D8F650B5EFC20E9] test/(1/1) ...m/testFilterAll/filterall
2016-01-14 15:21:48,725 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452784908580_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0C0F02ECA88248028E1309DD30ED57C3/DBBAC2516DE440508D8F650B5EFC20E9/.tez/application_1452784908580_0001/localmode-log-dir/dag_1452784908580_0001_1.dot
2016-01-14 15:21:48,725 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0C0F02ECA88248028E1309DD30ED57C3/DBBAC2516DE440508D8F650B5EFC20E9/.tez/application_1452784908580_0001/localmode-log-dir/dag_1452784908580_0001_1-tez-dag.pb.txt
2016-01-14 15:21:48,732 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784908580_0001_1
2016-01-14 15:21:48,732 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:48,732 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [0C0F02ECA88248028E1309DD30ED57C3/DBBAC2516DE440508D8F650B5EFC20E9] test/(1/1) ...m/testFilterAll/filterall
2016-01-14 15:21:48 Running Dag: dag_1452784908580_0001_1
2016-01-14 15:21:48,743 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784908580_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452784908580_0001_1, submitTime=1452784908724
2016-01-14 15:21:48,743 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:21:48,744 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D3769C5B2F8A42E3A086E54F0299CF52
2016-01-14 15:21:48,744 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:21:48,746 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:21:48,746 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784908580_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452784908580_0001_1, initTime=1452784908744
2016-01-14 15:21:48,746 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784908580_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:21:48,748 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784908580_0001_1][Event:DAG_STARTED]: dagID=dag_1452784908580_0001_1, startTime=1452784908748
2016-01-14 15:21:48,748 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:21:48,748 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784908580_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:21:48,749 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D3769C5B2F8A42E3A086E54F0299CF52 : {AA214D95035F41738D8C09534F3B08EB={InputName=AA214D95035F41738D8C09534F3B08EB}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:21:48,749 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: AA214D95035F41738D8C09534F3B08EB, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:21:48,749 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452784908580_0001_1_00 [D3769C5B2F8A42E3A086E54F0299CF52]
2016-01-14 15:21:48,749 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452784908580_0001_1_00 [D3769C5B2F8A42E3A086E54F0299CF52]
2016-01-14 15:21:48,749 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452784908580_0001_1_00 [D3769C5B2F8A42E3A086E54F0299CF52]
2016-01-14 15:21:48,750 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452784908580_0001_1_00 [D3769C5B2F8A42E3A086E54F0299CF52]. Starting root input initializers: 1
2016-01-14 15:21:48,750 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:21:48,749 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...m/testFilterAll/filterall, applicationId=application_1452784908580_0001, dagName=[0C0F02ECA88248028E1309DD30ED57C3/DBBAC2516DE440508D8F650B5EFC20E9] test/(1/1) ...m/testFilterAll/filterall
2016-01-14 15:21:48,752 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784908580_0001_1_00 [D3769C5B2F8A42E3A086E54F0299CF52] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:21:48,753 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: AA214D95035F41738D8C09534F3B08EB on vertex vertex_1452784908580_0001_1_00 [D3769C5B2F8A42E3A086E54F0299CF52]
2016-01-14 15:21:48,755 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:21:48,755 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input AA214D95035F41738D8C09534F3B08EB asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:21:48,756 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:21:48,757 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:21:48,760 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-01-14 15:21:48,760 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: AA214D95035F41738D8C09534F3B08EB on vertex vertex_1452784908580_0001_1_00 [D3769C5B2F8A42E3A086E54F0299CF52]
2016-01-14 15:21:48,761 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784908580_0001_1_00 [D3769C5B2F8A42E3A086E54F0299CF52] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:48,761 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {AA214D95035F41738D8C09534F3B08EB=forAllWorkUnits=true, update=[1]}
2016-01-14 15:21:48,761 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452784908580_0001_1_00 [D3769C5B2F8A42E3A086E54F0299CF52] parallelism set to 1
2016-01-14 15:21:48,762 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784908580_0001_1_00 [D3769C5B2F8A42E3A086E54F0299CF52] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:48,762 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784908580_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784908580_0001_1_00, eventCount=1
2016-01-14 15:21:48,762 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452784908580_0001_1_00 [D3769C5B2F8A42E3A086E54F0299CF52]
2016-01-14 15:21:48,762 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=90621AB0B2B44400B19F93E38E99324D, vertexId=vertex_1452784908580_0001_1_00 [D3769C5B2F8A42E3A086E54F0299CF52], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:21:48,762 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=90621AB0B2B44400B19F93E38E99324D, vertexId=vertex_1452784908580_0001_1_00 [D3769C5B2F8A42E3A086E54F0299CF52]
2016-01-14 15:21:48,765 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=90621AB0B2B44400B19F93E38E99324D, vertexName=D3769C5B2F8A42E3A086E54F0299CF52, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:21:48,765 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=90621AB0B2B44400B19F93E38E99324D, vertexId=vertex_1452784908580_0001_1_00 [D3769C5B2F8A42E3A086E54F0299CF52]
2016-01-14 15:21:48,767 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:48,767 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784908580_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D3769C5B2F8A42E3A086E54F0299CF52, vertexId=vertex_1452784908580_0001_1_00, initRequestedTime=1452784908749, initedTime=1452784908767, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:21:48,767 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784908580_0001_1_00 [D3769C5B2F8A42E3A086E54F0299CF52] with distanceFromRoot: 0
2016-01-14 15:21:48,767 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784908580_0001_1_00 [D3769C5B2F8A42E3A086E54F0299CF52] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:21:48,767 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D3769C5B2F8A42E3A086E54F0299CF52
2016-01-14 15:21:48,768 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784908580_0001_1_00 [D3769C5B2F8A42E3A086E54F0299CF52]
2016-01-14 15:21:48,768 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:48,768 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784908580_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784908580_0001_1_00, startRequestedTime=1452784908767, startedTime=1452784908767
2016-01-14 15:21:48,768 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784908580_0001_1_00 [D3769C5B2F8A42E3A086E54F0299CF52] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:48,769 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:48,769 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784908580_0001_1][Event:TASK_STARTED]: vertexName=D3769C5B2F8A42E3A086E54F0299CF52, taskId=task_1452784908580_0001_1_00_000000, scheduledTime=1452784908768, launchTime=1452784908768
2016-01-14 15:21:48,769 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784908580_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:48,769 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784908580_0001_1_00 [D3769C5B2F8A42E3A086E54F0299CF52]
2016-01-14 15:21:48,770 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [0C0F02ECA88248028E1309DD30ED57C3/DBBAC2516DE440508D8F650B5EFC20E9] test/(1/1) ...m/testFilterAll/filterall, VertexName: D3769C5B2F8A42E3A086E54F0299CF52, VertexParallelism: 1, TaskAttemptID:attempt_1452784908580_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AA214D95035F41738D8C09534F3B08EB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=90621AB0B2B44400B19F93E38E99324D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:48,770 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784908580_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:48,770 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:48,775 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:21:48,776 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:21:48,776 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:21:48,776 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784908580_0001_00_000001
2016-01-14 15:21:48,777 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784908580_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:48,778 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784908580_0001_00_000001
2016-01-14 15:21:48,778 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:21:48,783 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452784908580_0001, with dag id: dag_1452784908580_0001_1
2016-01-14 15:21:48,783 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:48,783 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784908580_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:48,783 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784908580_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784908580_0001_00_000001, launchTime=1452784908783
2016-01-14 15:21:48,789 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784908580_0001_00_000001
2016-01-14 15:21:48,793 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784908580_0001_1_00_000000_0] to container: [container_1452784908580_0001_00_000001]
2016-01-14 15:21:48,793 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784908580_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:48,794 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784908580_0001_1_00_000000_0] started. Is using containerId: [container_1452784908580_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:21:48,794 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:48,794 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784908580_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D3769C5B2F8A42E3A086E54F0299CF52, taskAttemptId=attempt_1452784908580_0001_1_00_000000_0, startTime=1452784908794, containerId=container_1452784908580_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784908580_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:21:48,794 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784908580_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:48,795 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784908580_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:48,797 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784908580_0001_00_000001 given task: attempt_1452784908580_0001_1_00_000000_0
2016-01-14 15:21:48,797 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784908580_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784908580_0001_1_00_000000_0
2016-01-14 15:21:48 Starting to run new task attempt: attempt_1452784908580_0001_1_00_000000_0
2016-01-14 15:21:48,797 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784908580_0001_1_00_000000_0
2016-01-14 15:21:48,798 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:48,798 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:48,798 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:48,798 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:48,798 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:48,798 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:48,798 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [0C0F02ECA88248028E1309DD30ED57C3/DBBAC2516DE440508D8F650B5EFC20E9] test/(1/1) ...m/testFilterAll/filterall, VertexName: D3769C5B2F8A42E3A086E54F0299CF52, VertexParallelism: 1, TaskAttemptID:attempt_1452784908580_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AA214D95035F41738D8C09534F3B08EB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=90621AB0B2B44400B19F93E38E99324D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:48,799 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:48,801 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784908580_0001_1_00_000000_0
2016-01-14 15:21:48,801 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:48,801 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:48,803 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=AA214D95035F41738D8C09534F3B08EB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:21:48,803 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:48,804 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: AA214D95035F41738D8C09534F3B08EB
2016-01-14 15:21:48,804 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: AA214D95035F41738D8C09534F3B08EB
2016-01-14 15:21:48,806 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:48,806 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=90621AB0B2B44400B19F93E38E99324D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:21:48,807 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:48,807 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 90621AB0B2B44400B19F93E38E99324D
2016-01-14 15:21:48,807 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:21:48,807 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 90621AB0B2B44400B19F93E38E99324D
2016-01-14 15:21:48,812 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:48,816 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784908580_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:21:48,817 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:21:48,821 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: AA214D95035F41738D8C09534F3B08EB
2016-01-14 15:21:48,823 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: AA214D95035F41738D8C09534F3B08EB
2016-01-14 15:21:48,823 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D3769C5B2F8A42E3A086E54F0299CF52, ordinal: 0
2016-01-14 15:21:48,823 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D3769C5B2F8A42E3A086E54F0299CF52, mem on start (mb), free: 443, total: 517, max: 672
2016-01-14 15:21:48,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:48,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:48,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:48,830 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:21:48,830 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 90621AB0B2B44400B19F93E38E99324D
2016-01-14 15:21:48,830 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:48,830 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:48,830 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:21:48,831 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:48,831 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:21:48,831 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:48,831 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:21:48,831 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-01-14 15:21:48,831 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, AA214D95035F41738D8C09534F3B08EB, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:21:48,831 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 90621AB0B2B44400B19F93E38E99324D, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:21:48,832 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:48,832 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: AA214D95035F41738D8C09534F3B08EB being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:48,832 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:21:48,832 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:48,832 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: AA214D95035F41738D8C09534F3B08EB
2016-01-14 15:21:48,832 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:48,832 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: AA214D95035F41738D8C09534F3B08EB
2016-01-14 15:21:48,836 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:48,836 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784908580_0001_1_00_000000_0
2016-01-14 15:21:48,839 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 15:21:48,840 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: AA214D95035F41738D8C09534F3B08EB
2016-01-14 15:21:48,840 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall"], id: 90621AB0B2B44400B19F93E38E99324D
2016-01-14 15:21:48,840 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:21:48,841 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6ccaffb3
2016-01-14 15:21:48,841 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall"]
2016-01-14 15:21:48,843 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 15:21:48,843 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:48,845 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6ccaffb3
2016-01-14 15:21:48,845 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:21:48,845 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:21:48,846 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D3769C5B2F8A42E3A086E54F0299CF52, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:21:48,848 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:21:48,848 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:21:48,849 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452784908580_0001_1_00_000000_0
2016-01-14 15:21:48,849 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452784908580_0001_1_00_000000_0 given a go for committing the task output.
2016-01-14 15:21:48,851 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784908580_0001_1_00_000000_0
2016-01-14 15:21:48,851 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D3769C5B2F8A42E3A086E54F0299CF52, mem on close (mb), free: 442, total: 517, max: 672
2016-01-14 15:21:48 Completed running task attempt: attempt_1452784908580_0001_1_00_000000_0
2016-01-14 15:21:48,852 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784908580_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:48,853 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:48,853 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784908580_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D3769C5B2F8A42E3A086E54F0299CF52, taskAttemptId=attempt_1452784908580_0001_1_00_000000_0, startTime=1452784908794, finishTime=1452784908853, timeTaken=59, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=542638080, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784908846, Process_Duration=5, Process_End_Time=1452784908851, Read_Duration=2, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10
2016-01-14 15:21:48,853 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784908580_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:48,853 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:48,853 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784908580_0001_1_00_000000_0
2016-01-14 15:21:48,854 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:48,854 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784908580_0001_1][Event:TASK_FINISHED]: vertexName=D3769C5B2F8A42E3A086E54F0299CF52, taskId=task_1452784908580_0001_1_00_000000, startTime=1452784908794, finishTime=1452784908854, timeTaken=60, status=SUCCEEDED, successfulAttemptID=attempt_1452784908580_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=542638080, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784908846, Process_Duration=5, Process_End_Time=1452784908851, Read_Duration=2, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10
2016-01-14 15:21:48,854 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784908580_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:48,855 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784908580_0001_1_00 [D3769C5B2F8A42E3A086E54F0299CF52] : 1
2016-01-14 15:21:48,855 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784908580_0001_1_00 [D3769C5B2F8A42E3A086E54F0299CF52]
2016-01-14 15:21:48,856 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:48,856 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784908580_0001_1][Event:VERTEX_FINISHED]: vertexName=D3769C5B2F8A42E3A086E54F0299CF52, vertexId=vertex_1452784908580_0001_1_00, initRequestedTime=1452784908749, initedTime=1452784908767, startRequestedTime=1452784908767, startedTime=1452784908767, finishTime=1452784908855, timeTaken=88, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=542638080, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784908846, Process_Duration=5, Process_End_Time=1452784908851, Read_Duration=2, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1452784908794, firstTasksToStart=[ task_1452784908580_0001_1_00_000000 ], lastTaskFinishTime=1452784908853, lastTasksToFinish=[ task_1452784908580_0001_1_00_000000 ], minTaskDuration=59, maxTaskDuration=59, avgTaskDuration=59.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784908580_0001_1_00_000000 ], longestDurationTasks=[ task_1452784908580_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:48,856 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784908580_0001_1_00 [D3769C5B2F8A42E3A086E54F0299CF52] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:48,857 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784908580_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:48,857 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784908580_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784908580_0001_00_000001, stoppedTime=1452784908857, exitStatus=0
2016-01-14 15:21:48,857 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:48,857 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784908580_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:48,857 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784908580_0001_1_00 [D3769C5B2F8A42E3A086E54F0299CF52] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-01-14 15:21:48,858 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452784908580_0001_1
2016-01-14 15:21:48,858 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784908580_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452784908580_0001_1
2016-01-14 15:21:48,858 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 90621AB0B2B44400B19F93E38E99324D for vertex: vertex_1452784908580_0001_1_00 [D3769C5B2F8A42E3A086E54F0299CF52]
2016-01-14 15:21:48,859 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 19 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=542638080, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1452784908846, Process_Duration=5, Process_End_Time=1452784908851, Read_Duration=2, Tuples_Read=10][cascading.flow.StepCounters Tuples_Read=10]]
2016-01-14 15:21:48,861 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:48,861 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:48,862 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784908580_0001_00_000001
2016-01-14 15:21:48,862 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784908580_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:21:48,863 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784908580_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:48,863 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784908580_0001_00_000001, Exiting
2016-01-14 15:21:48,864 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784908580_0001_00_000001
2016-01-14 15:21:48,864 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784908580_0001_00_000001
2016-01-14 15:21:48,870 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784908580_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:48,871 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784908580_0001_00_000001 completed successfully
2016-01-14 15:21:48,873 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452784908580_0001_1, queueSize=0
2016-01-14 15:21:48,874 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:21:48,874 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784908580_0001_1][Event:DAG_FINISHED]: dagId=dag_1452784908580_0001_1, startTime=1452784908748, finishTime=1452784908872, timeTaken=124, status=SUCCEEDED, diagnostics=, counters=Counters: 21, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=542638080, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784908846, Process_Duration=5, Process_End_Time=1452784908851, Read_Duration=2, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10
2016-01-14 15:21:48,874 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452784908580_0001_1 finished with state: SUCCEEDED
2016-01-14 15:21:48,874 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784908580_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:21:48,874 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784908580_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:21:48 Completed Dag: dag_1452784908580_0001_1
2016-01-14 15:21:48,875 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452784908580_0001_1, dagState=SUCCEEDED
2016-01-14 15:21:48,875 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784908580_0001_1_post
2016-01-14 15:21:48,875 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:48,875 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:21:48,875 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452784908580_0001_00_000001
2016-01-14 15:21:48,882 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...m/testFilterAll/filterall, applicationId=application_1452784908580_0001
2016-01-14 15:21:48,884 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:21:48,884 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:21:48,884 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:21:48,884 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:21:48,884 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:21:48,885 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:21:48,885 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:21:48,885 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:21:48,886 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:21:48,886 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:21:48,886 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:21:48,886 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:21:48,886 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:21:48,893 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0C0F02ECA88248028E1309DD30ED57C3/DBBAC2516DE440508D8F650B5EFC20E9/.tez/application_1452784908580_0001
2016-01-14 15:21:48,894 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:00.333
2016-01-14 15:21:48,927 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testFilterAll/filterall
2016-01-14 15:21:48,962 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:21:48,963 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:21:49,009 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-01-14 15:21:49,029 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.036
2016-01-14 15:21:49,030 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:21:49,030 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:21:49,030 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:21:49,059 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt
2016-01-14 15:21:49,069 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown
2016-01-14 15:21:49,121 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 15:21:49,121 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 15:21:49,122 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown"]
2016-01-14 15:21:49,122 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 15:21:49,123 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 15:21:49,123 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 15:21:49,126 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...own/simplerelativeunknown
2016-01-14 15:21:49,126 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:21:49,139 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 15:21:49,143 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:49,144 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:21:49,145 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:21:49,154 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2BDCF1D848974A8DADC24D68F8CF89A3/C9D5956876234B3899D3CDD678BDD690/.tez/application_1452784909144_0001 doesn't exist and is created
2016-01-14 15:21:49,170 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:21:49,170 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:21:49,170 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2BDCF1D848974A8DADC24D68F8CF89A3/C9D5956876234B3899D3CDD678BDD690/.tez/application_1452784909144_0001
2016-01-14 15:21:49,175 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452784909144_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:49,186 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:21:49,188 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:21:49,190 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:21:49,190 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:21:49,191 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2BDCF1D848974A8DADC24D68F8CF89A3/C9D5956876234B3899D3CDD678BDD690/.tez/application_1452784909144_0001/localmode-log-dir
2016-01-14 15:21:49,192 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2BDCF1D848974A8DADC24D68F8CF89A3/C9D5956876234B3899D3CDD678BDD690/.tez/application_1452784909144_0001/localmode-log-dir/history.txt.appattempt_1452784909144_0001_000000, maxErrors=10
2016-01-14 15:21:49,192 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:21:49,194 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452784909144_0001_000000, appSubmitTime=1452784909175, launchTime=1452784909175
2016-01-14 15:21:49,199 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:21:49,200 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:21:49,201 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:21:49,201 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:21:49,201 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:34944
2016-01-14 15:21:49,207 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:21:49,209 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452784909144_0001_000000, startTime=1452784909209
2016-01-14 15:21:49,209 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:21:49,209 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:21:49,250 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:21:49,251 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:21:49,270 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:21:49,270 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:21:49,271 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...own/simplerelativeunknown, applicationId=application_1452784909144_0001, dagName=[2BDCF1D848974A8DADC24D68F8CF89A3/C9D5956876234B3899D3CDD678BDD690] test/(1/1) ...own/simplerelativeunknown
2016-01-14 15:21:49,287 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [2BDCF1D848974A8DADC24D68F8CF89A3/C9D5956876234B3899D3CDD678BDD690] test/(1/1) ...own/simplerelativeunknown
2016-01-14 15:21:49,288 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452784909144_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2BDCF1D848974A8DADC24D68F8CF89A3/C9D5956876234B3899D3CDD678BDD690/.tez/application_1452784909144_0001/localmode-log-dir/dag_1452784909144_0001_1.dot
2016-01-14 15:21:49,289 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2BDCF1D848974A8DADC24D68F8CF89A3/C9D5956876234B3899D3CDD678BDD690/.tez/application_1452784909144_0001/localmode-log-dir/dag_1452784909144_0001_1-tez-dag.pb.txt
2016-01-14 15:21:49,302 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784909144_0001_1
2016-01-14 15:21:49,302 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:49,302 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [2BDCF1D848974A8DADC24D68F8CF89A3/C9D5956876234B3899D3CDD678BDD690] test/(1/1) ...own/simplerelativeunknown
2016-01-14 15:21:49 Running Dag: dag_1452784909144_0001_1
2016-01-14 15:21:49,316 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909144_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452784909144_0001_1, submitTime=1452784909287
2016-01-14 15:21:49,316 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:21:49,318 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FADA233C773E4668A9E08FF94DCEEB72
2016-01-14 15:21:49,319 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:21:49,320 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909144_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452784909144_0001_1, initTime=1452784909317
2016-01-14 15:21:49,320 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:21:49,320 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784909144_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:21:49,322 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909144_0001_1][Event:DAG_STARTED]: dagID=dag_1452784909144_0001_1, startTime=1452784909322
2016-01-14 15:21:49,322 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:21:49,322 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:21:49,322 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784909144_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:21:49,322 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CA3AE73E941C4DC5BEA245CB9C977926 : {0CA3303966F145A3A9D647291110F9E7={InputName=0CA3303966F145A3A9D647291110F9E7}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:21:49,323 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 0CA3303966F145A3A9D647291110F9E7, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:21:49,323 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452784909144_0001_1_00 [CA3AE73E941C4DC5BEA245CB9C977926]
2016-01-14 15:21:49,323 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452784909144_0001_1_00 [CA3AE73E941C4DC5BEA245CB9C977926]
2016-01-14 15:21:49,323 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452784909144_0001_1_00 [CA3AE73E941C4DC5BEA245CB9C977926]
2016-01-14 15:21:49,323 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...own/simplerelativeunknown, applicationId=application_1452784909144_0001, dagName=[2BDCF1D848974A8DADC24D68F8CF89A3/C9D5956876234B3899D3CDD678BDD690] test/(1/1) ...own/simplerelativeunknown
2016-01-14 15:21:49,324 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452784909144_0001_1_00 [CA3AE73E941C4DC5BEA245CB9C977926]. Starting root input initializers: 1
2016-01-14 15:21:49,324 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784909144_0001_1_00 [CA3AE73E941C4DC5BEA245CB9C977926] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:21:49,324 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 0CA3303966F145A3A9D647291110F9E7 on vertex vertex_1452784909144_0001_1_00 [CA3AE73E941C4DC5BEA245CB9C977926]
2016-01-14 15:21:49,324 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452784909144_0001_1_01 [FADA233C773E4668A9E08FF94DCEEB72]
2016-01-14 15:21:49,328 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:21:49,328 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452784909144_0001_1_01 [FADA233C773E4668A9E08FF94DCEEB72]
2016-01-14 15:21:49,328 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452784909144_0001_1_01 [FADA233C773E4668A9E08FF94DCEEB72]
2016-01-14 15:21:49,329 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452784909144_0001_1_01 [FADA233C773E4668A9E08FF94DCEEB72]
2016-01-14 15:21:49,329 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=0644117F808A454FB4FB98E74289AC98, vertexId=vertex_1452784909144_0001_1_01 [FADA233C773E4668A9E08FF94DCEEB72], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:21:49,329 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=0644117F808A454FB4FB98E74289AC98, vertexId=vertex_1452784909144_0001_1_01 [FADA233C773E4668A9E08FF94DCEEB72]
2016-01-14 15:21:49,330 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:21:49,331 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 0CA3303966F145A3A9D647291110F9E7 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:21:49,331 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=0644117F808A454FB4FB98E74289AC98, vertexName=FADA233C773E4668A9E08FF94DCEEB72, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:21:49,332 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:21:49,333 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:21:49,334 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=0644117F808A454FB4FB98E74289AC98, vertexId=vertex_1452784909144_0001_1_01 [FADA233C773E4668A9E08FF94DCEEB72]
2016-01-14 15:21:49,336 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-01-14 15:21:49,336 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 0CA3303966F145A3A9D647291110F9E7 on vertex vertex_1452784909144_0001_1_00 [CA3AE73E941C4DC5BEA245CB9C977926]
2016-01-14 15:21:49,337 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:49,337 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909144_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FADA233C773E4668A9E08FF94DCEEB72, vertexId=vertex_1452784909144_0001_1_01, initRequestedTime=1452784909324, initedTime=1452784909337, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:21:49,337 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784909144_0001_1_01 [FADA233C773E4668A9E08FF94DCEEB72] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:21:49,338 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784909144_0001_1_00 [CA3AE73E941C4DC5BEA245CB9C977926] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:49,338 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {0CA3303966F145A3A9D647291110F9E7=forAllWorkUnits=true, update=[1]}
2016-01-14 15:21:49,338 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452784909144_0001_1_00 [CA3AE73E941C4DC5BEA245CB9C977926] parallelism set to 1
2016-01-14 15:21:49,338 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784909144_0001_1_00 [CA3AE73E941C4DC5BEA245CB9C977926] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:49,339 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909144_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784909144_0001_1_00, eventCount=1
2016-01-14 15:21:49,340 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:49,340 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909144_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CA3AE73E941C4DC5BEA245CB9C977926, vertexId=vertex_1452784909144_0001_1_00, initRequestedTime=1452784909322, initedTime=1452784909339, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:21:49,340 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784909144_0001_1_00 [CA3AE73E941C4DC5BEA245CB9C977926] with distanceFromRoot: 0
2016-01-14 15:21:49,341 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784909144_0001_1_00 [CA3AE73E941C4DC5BEA245CB9C977926] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:21:49,341 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CA3AE73E941C4DC5BEA245CB9C977926
2016-01-14 15:21:49,341 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784909144_0001_1_00 [CA3AE73E941C4DC5BEA245CB9C977926]
2016-01-14 15:21:49,342 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:49,342 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909144_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784909144_0001_1_00, startRequestedTime=1452784909341, startedTime=1452784909341
2016-01-14 15:21:49,342 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452784909144_0001, with dag id: dag_1452784909144_0001_1
2016-01-14 15:21:49,342 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784909144_0001_1_00 [CA3AE73E941C4DC5BEA245CB9C977926] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:49,345 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909144_0001_1][Event:TASK_STARTED]: vertexName=CA3AE73E941C4DC5BEA245CB9C977926, taskId=task_1452784909144_0001_1_00_000000, scheduledTime=1452784909345, launchTime=1452784909345
2016-01-14 15:21:49,345 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784909144_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:49,345 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452784909144_0001_1_00 for vertex: vertex_1452784909144_0001_1_01 [FADA233C773E4668A9E08FF94DCEEB72] numStartedSources: 1 numSources: 1
2016-01-14 15:21:49,346 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784909144_0001_1_01 [FADA233C773E4668A9E08FF94DCEEB72] with distanceFromRoot: 1
2016-01-14 15:21:49,345 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:49,347 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784909144_0001_1_00 [CA3AE73E941C4DC5BEA245CB9C977926]
2016-01-14 15:21:49,347 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: CA3AE73E941C4DC5BEA245CB9C977926 in vertex: FADA233C773E4668A9E08FF94DCEEB72
2016-01-14 15:21:49,347 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: FADA233C773E4668A9E08FF94DCEEB72 with 1 source tasks and 1 pending tasks
2016-01-14 15:21:49,348 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:49,349 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909144_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784909144_0001_1_01, startRequestedTime=1452784909347, startedTime=1452784909347
2016-01-14 15:21:49,349 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784909144_0001_1_01 [FADA233C773E4668A9E08FF94DCEEB72] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:49,349 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2BDCF1D848974A8DADC24D68F8CF89A3/C9D5956876234B3899D3CDD678BDD690] test/(1/1) ...own/simplerelativeunknown, VertexName: CA3AE73E941C4DC5BEA245CB9C977926, VertexParallelism: 1, TaskAttemptID:attempt_1452784909144_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0CA3303966F145A3A9D647291110F9E7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FADA233C773E4668A9E08FF94DCEEB72, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:21:49,349 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784909144_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:49,350 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:49,351 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:21:49,351 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:21:49,352 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:21:49,352 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784909144_0001_00_000001
2016-01-14 15:21:49,353 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909144_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:49,355 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784909144_0001_00_000001
2016-01-14 15:21:49,355 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:21:49,356 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:49,356 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909144_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784909144_0001_00_000001, launchTime=1452784909356
2016-01-14 15:21:49,357 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784909144_0001_00_000001
2016-01-14 15:21:49,357 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909144_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:49,358 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784909144_0001_1_00_000000_0] to container: [container_1452784909144_0001_00_000001]
2016-01-14 15:21:49,358 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909144_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:49,359 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784909144_0001_00_000001 given task: attempt_1452784909144_0001_1_00_000000_0
2016-01-14 15:21:49,359 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784909144_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784909144_0001_1_00_000000_0
2016-01-14 15:21:49 Starting to run new task attempt: attempt_1452784909144_0001_1_00_000000_0
2016-01-14 15:21:49,360 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784909144_0001_1_00_000000_0] started. Is using containerId: [container_1452784909144_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:21:49,360 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784909144_0001_1_00_000000_0
2016-01-14 15:21:49,360 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:49,360 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:49,360 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:49,360 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:49,361 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:49,361 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:49,361 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2BDCF1D848974A8DADC24D68F8CF89A3/C9D5956876234B3899D3CDD678BDD690] test/(1/1) ...own/simplerelativeunknown, VertexName: CA3AE73E941C4DC5BEA245CB9C977926, VertexParallelism: 1, TaskAttemptID:attempt_1452784909144_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0CA3303966F145A3A9D647291110F9E7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FADA233C773E4668A9E08FF94DCEEB72, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:21:49,361 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:49,362 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:49,362 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909144_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CA3AE73E941C4DC5BEA245CB9C977926, taskAttemptId=attempt_1452784909144_0001_1_00_000000_0, startTime=1452784909360, containerId=container_1452784909144_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784909144_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:21:49,362 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784909144_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:49,362 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784909144_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:49,363 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784909144_0001_1_00_000000_0
2016-01-14 15:21:49,363 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:49,364 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:49,363 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784909144_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:21:49,368 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=0CA3303966F145A3A9D647291110F9E7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:21:49,369 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:49,369 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:49,370 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FADA233C773E4668A9E08FF94DCEEB72, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:21:49,371 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:49,371 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FADA233C773E4668A9E08FF94DCEEB72
2016-01-14 15:21:49,372 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:49,373 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:21:49,373 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: FADA233C773E4668A9E08FF94DCEEB72
2016-01-14 15:21:49,373 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FADA233C773E4668A9E08FF94DCEEB72
2016-01-14 15:21:49,374 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 0CA3303966F145A3A9D647291110F9E7
2016-01-14 15:21:49,374 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 0CA3303966F145A3A9D647291110F9E7
2016-01-14 15:21:49,378 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:21:49,378 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 0CA3303966F145A3A9D647291110F9E7
2016-01-14 15:21:49,378 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 0CA3303966F145A3A9D647291110F9E7
2016-01-14 15:21:49,382 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CA3AE73E941C4DC5BEA245CB9C977926, ordinal: 0
2016-01-14 15:21:49,383 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CA3AE73E941C4DC5BEA245CB9C977926, mem on start (mb), free: 445, total: 517, max: 672
2016-01-14 15:21:49,383 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:49,383 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:49,383 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:49,383 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:49,383 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:49,384 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:21:49,384 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:21:49,384 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:49,384 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:21:49,384 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:21:49,384 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FADA233C773E4668A9E08FF94DCEEB72, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:21:49,385 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 0CA3303966F145A3A9D647291110F9E7, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:21:49,385 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:49,385 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 0CA3303966F145A3A9D647291110F9E7 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:49,385 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:21:49,385 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:49,385 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 0CA3303966F145A3A9D647291110F9E7
2016-01-14 15:21:49,386 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:49,386 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 0CA3303966F145A3A9D647291110F9E7
2016-01-14 15:21:49,386 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:49,386 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784909144_0001_1_00_000000_0
2016-01-14 15:21:49,388 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:21:49,388 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 15:21:49,389 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"] streamed: true, id: 0CA3303966F145A3A9D647291110F9E7
2016-01-14 15:21:49,389 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:-1]], id: FF5B5CB2EAEC466BA9145F5BD73426AB
2016-01-14 15:21:49,389 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:-1]] FF5B5CB2EAEC466BA9145F5BD73426AB
2016-01-14 15:21:49,389 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@10816c49
2016-01-14 15:21:49,390 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@7391483c; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@471d966a; comparator=cascading.tuple.hadoop.util.TupleComparator@677e2d9f; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:21:49,391 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:21:49,392 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:21:49,392 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@10816c49
2016-01-14 15:21:49,393 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:21:49,393 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:21:49,393 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:21:49,408 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:21:49,409 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:21:49,409 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:21:49,409 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:21:49,409 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:21:49,410 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 15:21:49,410 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:49,410 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CA3AE73E941C4DC5BEA245CB9C977926, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:21:49,412 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784909144_0001_1_00_000000_0
2016-01-14 15:21:49,413 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CA3AE73E941C4DC5BEA245CB9C977926, mem on close (mb), free: 343, total: 517, max: 672
2016-01-14 15:21:49,413 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:21:49,413 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:21:49,413 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-01-14 15:21:49,413 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-01-14 15:21:49,415 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:21:49 Completed running task attempt: attempt_1452784909144_0001_1_00_000000_0
2016-01-14 15:21:49,420 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784909144_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:49,421 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909144_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784909144_0001_1_00, eventCount=1
2016-01-14 15:21:49,421 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=647, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=542638080, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452784909410, Process_Duration=2, Process_End_Time=1452784909412, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-01-14 15:21:49,422 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909144_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CA3AE73E941C4DC5BEA245CB9C977926, taskAttemptId=attempt_1452784909144_0001_1_00_000000_0, startTime=1452784909360, finishTime=1452784909421, timeTaken=61, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=647, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=542638080, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784909410, Process_Duration=2, Process_End_Time=1452784909412, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 15:21:49,422 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784909144_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:49,422 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784909144_0001_1_00_000000_0
2016-01-14 15:21:49,422 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:49,421 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:49,424 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:49,425 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:49,426 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:49,426 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784909144_0001_00_000001
2016-01-14 15:21:49,424 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909144_0001_1][Event:TASK_FINISHED]: vertexName=CA3AE73E941C4DC5BEA245CB9C977926, taskId=task_1452784909144_0001_1_00_000000, startTime=1452784909360, finishTime=1452784909422, timeTaken=62, status=SUCCEEDED, successfulAttemptID=attempt_1452784909144_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=647, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=542638080, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784909410, Process_Duration=2, Process_End_Time=1452784909412, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 15:21:49,426 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784909144_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:49,427 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784909144_0001_1_00 [CA3AE73E941C4DC5BEA245CB9C977926] : 1
2016-01-14 15:21:49,427 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784909144_0001_1_00 [CA3AE73E941C4DC5BEA245CB9C977926]
2016-01-14 15:21:49,429 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:49,429 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909144_0001_1][Event:VERTEX_FINISHED]: vertexName=CA3AE73E941C4DC5BEA245CB9C977926, vertexId=vertex_1452784909144_0001_1_00, initRequestedTime=1452784909322, initedTime=1452784909339, startRequestedTime=1452784909341, startedTime=1452784909341, finishTime=1452784909427, timeTaken=86, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=647, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=542638080, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784909410, Process_Duration=2, Process_End_Time=1452784909412, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1452784909360, firstTasksToStart=[ task_1452784909144_0001_1_00_000000 ], lastTaskFinishTime=1452784909421, lastTasksToFinish=[ task_1452784909144_0001_1_00_000000 ], minTaskDuration=61, maxTaskDuration=61, avgTaskDuration=61.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784909144_0001_1_00_000000 ], longestDurationTasks=[ task_1452784909144_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:49,429 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784909144_0001_1_00 [CA3AE73E941C4DC5BEA245CB9C977926] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:49,429 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909144_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:49,429 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909144_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784909144_0001_00_000001, stoppedTime=1452784909429, exitStatus=0
2016-01-14 15:21:49,430 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909144_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:49,430 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452784909144_0001_1_01 [FADA233C773E4668A9E08FF94DCEEB72] attempt: attempt_1452784909144_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:21:49,430 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:49,430 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: FADA233C773E4668A9E08FF94DCEEB72
2016-01-14 15:21:49,430 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784909144_0001_1_01 [FADA233C773E4668A9E08FF94DCEEB72]
2016-01-14 15:21:49,430 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784909144_0001_1_00 [CA3AE73E941C4DC5BEA245CB9C977926] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:21:49,431 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452784909144_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:21:49,431 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909144_0001_1][Event:TASK_STARTED]: vertexName=FADA233C773E4668A9E08FF94DCEEB72, taskId=task_1452784909144_0001_1_01_000000, scheduledTime=1452784909431, launchTime=1452784909431
2016-01-14 15:21:49,431 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:49,432 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784909144_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:49,432 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784909144_0001_1_01 [FADA233C773E4668A9E08FF94DCEEB72]
2016-01-14 15:21:49,432 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909144_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:21:49,433 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2BDCF1D848974A8DADC24D68F8CF89A3/C9D5956876234B3899D3CDD678BDD690] test/(1/1) ...own/simplerelativeunknown, VertexName: FADA233C773E4668A9E08FF94DCEEB72, VertexParallelism: 1, TaskAttemptID:attempt_1452784909144_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CA3AE73E941C4DC5BEA245CB9C977926, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=0644117F808A454FB4FB98E74289AC98, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:49,433 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784909144_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:49,433 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:49,434 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784909144_0001_00_000002
2016-01-14 15:21:49,434 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909144_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:49,436 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784909144_0001_00_000002
2016-01-14 15:21:49,436 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:21:49,436 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784909144_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:21:49,437 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784909144_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:49,437 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:49,437 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784909144_0001_00_000001, Exiting
2016-01-14 15:21:49,437 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784909144_0001_00_000001
2016-01-14 15:21:49,437 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784909144_0001_00_000001
2016-01-14 15:21:49,437 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909144_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784909144_0001_00_000002, launchTime=1452784909437
2016-01-14 15:21:49,438 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909144_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:49,438 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784909144_0001_00_000002
2016-01-14 15:21:49,440 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784909144_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:49,440 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784909144_0001_00_000001 completed successfully
2016-01-14 15:21:49,441 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909144_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:21:49,440 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784909144_0001_1_01_000000_0] to container: [container_1452784909144_0001_00_000002]
2016-01-14 15:21:49,442 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909144_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:49,443 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784909144_0001_1_01_000000_0] started. Is using containerId: [container_1452784909144_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:21:49,443 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:49,443 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909144_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FADA233C773E4668A9E08FF94DCEEB72, taskAttemptId=attempt_1452784909144_0001_1_01_000000_0, startTime=1452784909443, containerId=container_1452784909144_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784909144_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:21:49,444 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784909144_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:49,444 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784909144_0001_00_000002 given task: attempt_1452784909144_0001_1_01_000000_0
2016-01-14 15:21:49,444 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784909144_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:49,444 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784909144_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784909144_0001_1_01_000000_0
2016-01-14 15:21:49 Starting to run new task attempt: attempt_1452784909144_0001_1_01_000000_0
2016-01-14 15:21:49,445 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784909144_0001_1_01_000000_0
2016-01-14 15:21:49,445 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:49,445 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:49,445 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:49,445 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:49,446 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:49,446 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:49,446 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2BDCF1D848974A8DADC24D68F8CF89A3/C9D5956876234B3899D3CDD678BDD690] test/(1/1) ...own/simplerelativeunknown, VertexName: FADA233C773E4668A9E08FF94DCEEB72, VertexParallelism: 1, TaskAttemptID:attempt_1452784909144_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CA3AE73E941C4DC5BEA245CB9C977926, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=0644117F808A454FB4FB98E74289AC98, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:49,446 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:49,447 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784909144_0001_1_01_000000_0
2016-01-14 15:21:49,447 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:49,449 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:49,448 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784909144_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:21:49,451 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=CA3AE73E941C4DC5BEA245CB9C977926, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:21:49,452 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:49,452 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:49,452 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: CA3AE73E941C4DC5BEA245CB9C977926
2016-01-14 15:21:49,454 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:21:49,454 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:21:49,454 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: CA3AE73E941C4DC5BEA245CB9C977926
2016-01-14 15:21:49,454 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: CA3AE73E941C4DC5BEA245CB9C977926
2016-01-14 15:21:49,453 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=0644117F808A454FB4FB98E74289AC98, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:21:49,455 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:49,456 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:49,457 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 0644117F808A454FB4FB98E74289AC98
2016-01-14 15:21:49,457 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:21:49,457 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 0644117F808A454FB4FB98E74289AC98
2016-01-14 15:21:49,464 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FADA233C773E4668A9E08FF94DCEEB72, ordinal: 1
2016-01-14 15:21:49,464 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FADA233C773E4668A9E08FF94DCEEB72, mem on start (mb), free: 338, total: 517, max: 672
2016-01-14 15:21:49,464 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:49,465 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:49,465 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:49,470 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:21:49,470 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 0644117F808A454FB4FB98E74289AC98
2016-01-14 15:21:49,470 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:49,470 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:49,472 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:21:49,472 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:21:49,472 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:49,472 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:21:49,473 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:21:49,473 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, CA3AE73E941C4DC5BEA245CB9C977926, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:21:49,474 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 0644117F808A454FB4FB98E74289AC98, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:21:49,474 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:49,474 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: CA3AE73E941C4DC5BEA245CB9C977926
2016-01-14 15:21:49,474 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: CA3AE73E941C4DC5BEA245CB9C977926 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:49,475 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:21:49,475 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:21:49,476 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: CA3AE73E941C4DC5BEA245CB9C977926 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:21:49,476 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:21:49,476 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:21:49,476 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:21:49,476 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:21:49,476 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:49,479 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: CA3AE73E941C4DC5BEA245CB9C977926
2016-01-14 15:21:49,479 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:49,481 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784909144_0001_1_01_000000_0
2016-01-14 15:21:49,487 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452784909144_0001_1_00_000000_0_10049, runDuration: 45831, ]
2016-01-14 15:21:49,488 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:-1]]
2016-01-14 15:21:49,488 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452784909144_0001_1_00_000000_0_10049] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1452784909488, TimeTaken=1, Rate=0.09 MB/s
2016-01-14 15:21:49,488 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : CA3AE73E941C4DC5BEA245CB9C977926
2016-01-14 15:21:49,489 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:21:49,489 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:-1]] streamed: true, id: FF5B5CB2EAEC466BA9145F5BD73426AB
2016-01-14 15:21:49,490 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [CA3AE73E941C4DC5BEA245CB9C977926] #8 in 3ms
2016-01-14 15:21:49,490 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown"], id: 0644117F808A454FB4FB98E74289AC98
2016-01-14 15:21:49,490 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [CA3AE73E941C4DC5BEA245CB9C977926] #8, status:false, isInterrupted:false
2016-01-14 15:21:49,491 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown"]
2016-01-14 15:21:49,491 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:-1]] FF5B5CB2EAEC466BA9145F5BD73426AB, for 1 inputs
2016-01-14 15:21:49,491 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:21:49,492 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-01-14 15:21:49,492 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:21:49,492 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:21:49,492 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-01-14 15:21:49,492 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : CA3AE73E941C4DC5BEA245CB9C977926
2016-01-14 15:21:49,493 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:21:49,493 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FADA233C773E4668A9E08FF94DCEEB72, all 1 inputs ready in: 00:00:00.002
2016-01-14 15:21:49,495 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@219da9b; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@6f5ef730
2016-01-14 15:21:49,496 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:21:49,496 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:21:49,497 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452784909144_0001_1_01_000000_0
2016-01-14 15:21:49,497 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452784909144_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:21:49,498 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784909144_0001_1_01_000000_0
2016-01-14 15:21:49,498 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FADA233C773E4668A9E08FF94DCEEB72, mem on close (mb), free: 336, total: 517, max: 672
2016-01-14 15:21:49,498 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: CA3AE73E941C4DC5BEA245CB9C977926
2016-01-14 15:21:49,498 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:21:49,499 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:21:49,499 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:21:49,499 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:21:49,499 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:21:49 Completed running task attempt: attempt_1452784909144_0001_1_01_000000_0
2016-01-14 15:21:49,500 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784909144_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:49,501 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=127, FILE_BYTES_WRITTEN=32, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=542638080, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=15, MERGE_PHASE_TIME=17, FIRST_EVENT_RECEIVED=12, LAST_EVENT_RECEIVED=12][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452784909493, Process_Duration=5, Process_End_Time=1452784909498, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-01-14 15:21:49,501 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909144_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FADA233C773E4668A9E08FF94DCEEB72, taskAttemptId=attempt_1452784909144_0001_1_01_000000_0, startTime=1452784909443, finishTime=1452784909501, timeTaken=58, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=127, FILE_BYTES_WRITTEN=32, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=542638080, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=15, MERGE_PHASE_TIME=17, FIRST_EVENT_RECEIVED=12, LAST_EVENT_RECEIVED=12, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784909493, Process_Duration=5, Process_End_Time=1452784909498, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-01-14 15:21:49,501 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:49,502 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784909144_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:49,502 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:49,502 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:49,502 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784909144_0001_1_01_000000_0
2016-01-14 15:21:49,502 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784909144_0001_00_000002
2016-01-14 15:21:49,502 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:49,503 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:49,503 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909144_0001_1][Event:TASK_FINISHED]: vertexName=FADA233C773E4668A9E08FF94DCEEB72, taskId=task_1452784909144_0001_1_01_000000, startTime=1452784909443, finishTime=1452784909502, timeTaken=59, status=SUCCEEDED, successfulAttemptID=attempt_1452784909144_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=127, FILE_BYTES_WRITTEN=32, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=542638080, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=15, MERGE_PHASE_TIME=17, FIRST_EVENT_RECEIVED=12, LAST_EVENT_RECEIVED=12, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784909493, Process_Duration=5, Process_End_Time=1452784909498, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-01-14 15:21:49,503 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784909144_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:49,503 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784909144_0001_1_01 [FADA233C773E4668A9E08FF94DCEEB72] : 1
2016-01-14 15:21:49,504 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784909144_0001_1_01 [FADA233C773E4668A9E08FF94DCEEB72]
2016-01-14 15:21:49,505 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:49,505 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909144_0001_1][Event:VERTEX_FINISHED]: vertexName=FADA233C773E4668A9E08FF94DCEEB72, vertexId=vertex_1452784909144_0001_1_01, initRequestedTime=1452784909324, initedTime=1452784909337, startRequestedTime=1452784909347, startedTime=1452784909347, finishTime=1452784909504, timeTaken=157, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=127, FILE_BYTES_WRITTEN=32, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=542638080, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=15, MERGE_PHASE_TIME=17, FIRST_EVENT_RECEIVED=12, LAST_EVENT_RECEIVED=12, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784909493, Process_Duration=5, Process_End_Time=1452784909498, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1452784909443, firstTasksToStart=[ task_1452784909144_0001_1_01_000000 ], lastTaskFinishTime=1452784909501, lastTasksToFinish=[ task_1452784909144_0001_1_01_000000 ], minTaskDuration=58, maxTaskDuration=58, avgTaskDuration=58.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784909144_0001_1_01_000000 ], longestDurationTasks=[ task_1452784909144_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:49,506 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784909144_0001_1_01 [FADA233C773E4668A9E08FF94DCEEB72] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:49,506 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909144_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:49,506 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909144_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784909144_0001_00_000002, stoppedTime=1452784909506, exitStatus=0
2016-01-14 15:21:49,506 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:49,507 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909144_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:49,508 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784909144_0001_1_01 [FADA233C773E4668A9E08FF94DCEEB72] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:21:49,508 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452784909144_0001_1
2016-01-14 15:21:49,508 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909144_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452784909144_0001_1
2016-01-14 15:21:49,508 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452784909144_0001_1_00 [CA3AE73E941C4DC5BEA245CB9C977926]
2016-01-14 15:21:49,509 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 0644117F808A454FB4FB98E74289AC98 for vertex: vertex_1452784909144_0001_1_01 [FADA233C773E4668A9E08FF94DCEEB72]
2016-01-14 15:21:49,513 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784909144_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:21:49,513 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784909144_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:49,513 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784909144_0001_00_000002, Exiting
2016-01-14 15:21:49,513 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784909144_0001_00_000002
2016-01-14 15:21:49,513 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784909144_0001_00_000002
2016-01-14 15:21:49,514 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784909144_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:49,514 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784909144_0001_00_000002 completed successfully
2016-01-14 15:21:49,523 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452784909144_0001_1, queueSize=0
2016-01-14 15:21:49,523 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:21:49,525 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909144_0001_1][Event:DAG_FINISHED]: dagId=dag_1452784909144_0001_1, startTime=1452784909322, finishTime=1452784909520, timeTaken=198, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=189, FILE_BYTES_WRITTEN=679, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1085276160, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=15, MERGE_PHASE_TIME=17, FIRST_EVENT_RECEIVED=12, LAST_EVENT_RECEIVED=12, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905569818903, Process_Duration=7, Process_End_Time=2905569818910, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-01-14 15:21:49,525 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452784909144_0001_1 finished with state: SUCCEEDED
2016-01-14 15:21:49,525 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784909144_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:21:49,525 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452784909144_0001_00_000002
2016-01-14 15:21:49,526 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909144_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:21:49 Completed Dag: dag_1452784909144_0001_1
2016-01-14 15:21:49,527 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452784909144_0001_1, dagState=SUCCEEDED
2016-01-14 15:21:49,527 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784909144_0001_1_post
2016-01-14 15:21:49,527 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:49,527 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:21:49,536 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...own/simplerelativeunknown, applicationId=application_1452784909144_0001
2016-01-14 15:21:49,538 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:21:49,538 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:21:49,538 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:21:49,538 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:21:49,538 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:21:49,539 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:21:49,541 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:21:49,541 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:21:49,539 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:21:49,542 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:21:49,543 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:21:49,543 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:21:49,543 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:21:49,549 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2BDCF1D848974A8DADC24D68F8CF89A3/C9D5956876234B3899D3CDD678BDD690/.tez/application_1452784909144_0001
2016-01-14 15:21:49,550 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:00.428
2016-01-14 15:21:49,583 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown
2016-01-14 15:21:49,610 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:21:49,610 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:21:49,661 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.025
2016-01-14 15:21:49,683 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.047
2016-01-14 15:21:49,684 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-01-14 15:21:49,684 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-01-14 15:21:49,684 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:21:49,702 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt
2016-01-14 15:21:49,708 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt
2016-01-14 15:21:49,715 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup
2016-01-14 15:21:49,803 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] starting
2016-01-14 15:21:49,803 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-01-14 15:21:49,804 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 15:21:49,804 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup"]
2016-01-14 15:21:49,804 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] parallel execution of steps is enabled: false
2016-01-14 15:21:49,804 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] executing total steps: 1
2016-01-14 15:21:49,805 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] allocating management threads: 1
2016-01-14 15:21:49,806 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] starting step: (1/1) ...estCoGroup/complexcogroup
2016-01-14 15:21:49,806 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [lower*upper] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:21:49,818 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] tez session mode enabled: true
2016-01-14 15:21:49,822 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:49,823 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:21:49,823 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:21:49,833 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8017674F884343859A63E3E0BA74A805/CD27E258D9E7405BA05B6426EE39338B/.tez/application_1452784909823_0001 doesn't exist and is created
2016-01-14 15:21:49,851 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:21:49,851 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:21:49,852 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8017674F884343859A63E3E0BA74A805/CD27E258D9E7405BA05B6426EE39338B/.tez/application_1452784909823_0001
2016-01-14 15:21:49,856 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452784909823_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:49,866 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:21:49,868 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:21:49,869 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:21:49,870 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:21:49,870 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8017674F884343859A63E3E0BA74A805/CD27E258D9E7405BA05B6426EE39338B/.tez/application_1452784909823_0001/localmode-log-dir
2016-01-14 15:21:49,871 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8017674F884343859A63E3E0BA74A805/CD27E258D9E7405BA05B6426EE39338B/.tez/application_1452784909823_0001/localmode-log-dir/history.txt.appattempt_1452784909823_0001_000000, maxErrors=10
2016-01-14 15:21:49,871 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:21:49,872 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452784909823_0001_000000, appSubmitTime=1452784909856, launchTime=1452784909856
2016-01-14 15:21:49,879 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:21:49,879 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:59351
2016-01-14 15:21:49,879 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:21:49,882 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:21:49,882 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:21:49,882 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:21:49,885 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452784909823_0001_000000, startTime=1452784909885
2016-01-14 15:21:49,885 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:21:49,885 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:21:49,892 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:21:49,892 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:21:49,952 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:21:49,952 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:21:49,952 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...estCoGroup/complexcogroup, applicationId=application_1452784909823_0001, dagName=[8017674F884343859A63E3E0BA74A805/CD27E258D9E7405BA05B6426EE39338B] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-01-14 15:21:49,970 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [8017674F884343859A63E3E0BA74A805/CD27E258D9E7405BA05B6426EE39338B] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-01-14 15:21:49,972 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452784909823_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8017674F884343859A63E3E0BA74A805/CD27E258D9E7405BA05B6426EE39338B/.tez/application_1452784909823_0001/localmode-log-dir/dag_1452784909823_0001_1.dot
2016-01-14 15:21:49,973 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8017674F884343859A63E3E0BA74A805/CD27E258D9E7405BA05B6426EE39338B/.tez/application_1452784909823_0001/localmode-log-dir/dag_1452784909823_0001_1-tez-dag.pb.txt
2016-01-14 15:21:49,982 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784909823_0001_1
2016-01-14 15:21:49,982 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:49,982 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [8017674F884343859A63E3E0BA74A805/CD27E258D9E7405BA05B6426EE39338B] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-01-14 15:21:49 Running Dag: dag_1452784909823_0001_1
2016-01-14 15:21:49,998 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:21:49,998 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452784909823_0001_1, submitTime=1452784909970
2016-01-14 15:21:49,999 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BA93E953F8744BC09E649F1F4D7F9C24
2016-01-14 15:21:50,000 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:21:50,000 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:21:50,000 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452784909823_0001_1, initTime=1452784909998
2016-01-14 15:21:50,001 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784909823_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:21:50,001 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:DAG_STARTED]: dagID=dag_1452784909823_0001_1, startTime=1452784910001
2016-01-14 15:21:50,002 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:21:50,002 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784909823_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:21:50,002 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: AD2E9C268989402CBB4848BC6BA7DFAF : {BA97675EAADF475DB321EDF1B0F7493F={InputName=BA97675EAADF475DB321EDF1B0F7493F}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:21:50,002 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: BA97675EAADF475DB321EDF1B0F7493F, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:21:50,002 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452784909823_0001_1_00 [AD2E9C268989402CBB4848BC6BA7DFAF]
2016-01-14 15:21:50,003 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452784909823_0001_1_00 [AD2E9C268989402CBB4848BC6BA7DFAF]
2016-01-14 15:21:50,003 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452784909823_0001_1_00 [AD2E9C268989402CBB4848BC6BA7DFAF]
2016-01-14 15:21:50,003 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...estCoGroup/complexcogroup, applicationId=application_1452784909823_0001, dagName=[8017674F884343859A63E3E0BA74A805/CD27E258D9E7405BA05B6426EE39338B] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-01-14 15:21:50,003 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452784909823_0001_1_00 [AD2E9C268989402CBB4848BC6BA7DFAF]. Starting root input initializers: 1
2016-01-14 15:21:50,001 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:21:50,010 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784909823_0001_1_00 [AD2E9C268989402CBB4848BC6BA7DFAF] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:21:50,010 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: BA97675EAADF475DB321EDF1B0F7493F on vertex vertex_1452784909823_0001_1_00 [AD2E9C268989402CBB4848BC6BA7DFAF]
2016-01-14 15:21:50,010 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F67FF5BE0D834E48919D8C9428108694 : {47A2B96A5AD848EEBBCFA7CE07AED287={InputName=47A2B96A5AD848EEBBCFA7CE07AED287}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:21:50,011 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 47A2B96A5AD848EEBBCFA7CE07AED287, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:21:50,011 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452784909823_0001_1_01 [F67FF5BE0D834E48919D8C9428108694]
2016-01-14 15:21:50,011 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452784909823_0001_1_01 [F67FF5BE0D834E48919D8C9428108694]
2016-01-14 15:21:50,011 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452784909823_0001_1_01 [F67FF5BE0D834E48919D8C9428108694]
2016-01-14 15:21:50,011 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452784909823_0001_1_01 [F67FF5BE0D834E48919D8C9428108694]. Starting root input initializers: 1
2016-01-14 15:21:50,016 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:21:50,017 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input BA97675EAADF475DB321EDF1B0F7493F asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:21:50,017 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784909823_0001_1_01 [F67FF5BE0D834E48919D8C9428108694] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:21:50,017 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452784909823_0001_1_02 [BA93E953F8744BC09E649F1F4D7F9C24]
2016-01-14 15:21:50,018 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 47A2B96A5AD848EEBBCFA7CE07AED287 on vertex vertex_1452784909823_0001_1_01 [F67FF5BE0D834E48919D8C9428108694]
2016-01-14 15:21:50,020 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:21:50,020 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 47A2B96A5AD848EEBBCFA7CE07AED287 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:21:50,020 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:21:50,021 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452784909823_0001_1_02 [BA93E953F8744BC09E649F1F4D7F9C24]
2016-01-14 15:21:50,021 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452784909823_0001_1_02 [BA93E953F8744BC09E649F1F4D7F9C24]
2016-01-14 15:21:50,021 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452784909823_0001_1_02 [BA93E953F8744BC09E649F1F4D7F9C24]
2016-01-14 15:21:50,022 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=31C2DBAB314E413BB81EB5A2C43198DB, vertexId=vertex_1452784909823_0001_1_02 [BA93E953F8744BC09E649F1F4D7F9C24], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:21:50,022 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=31C2DBAB314E413BB81EB5A2C43198DB, vertexId=vertex_1452784909823_0001_1_02 [BA93E953F8744BC09E649F1F4D7F9C24]
2016-01-14 15:21:50,028 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] submitted tez dag to app master: application_1452784909823_0001, with dag id: dag_1452784909823_0001_1
2016-01-14 15:21:50,028 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:21:50,029 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=31C2DBAB314E413BB81EB5A2C43198DB, vertexName=BA93E953F8744BC09E649F1F4D7F9C24, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:21:50,028 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:21:50,029 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:21:50,030 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:21:50,031 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=31C2DBAB314E413BB81EB5A2C43198DB, vertexId=vertex_1452784909823_0001_1_02 [BA93E953F8744BC09E649F1F4D7F9C24]
2016-01-14 15:21:50,033 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:50,033 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BA93E953F8744BC09E649F1F4D7F9C24, vertexId=vertex_1452784909823_0001_1_02, initRequestedTime=1452784910017, initedTime=1452784910033, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:21:50,034 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784909823_0001_1_02 [BA93E953F8744BC09E649F1F4D7F9C24] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:21:50,034 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-01-14 15:21:50,035 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 47A2B96A5AD848EEBBCFA7CE07AED287 on vertex vertex_1452784909823_0001_1_01 [F67FF5BE0D834E48919D8C9428108694]
2016-01-14 15:21:50,035 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784909823_0001_1_01 [F67FF5BE0D834E48919D8C9428108694] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:50,035 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {47A2B96A5AD848EEBBCFA7CE07AED287=forAllWorkUnits=true, update=[1]}
2016-01-14 15:21:50,035 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452784909823_0001_1_01 [F67FF5BE0D834E48919D8C9428108694] parallelism set to 1
2016-01-14 15:21:50,036 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784909823_0001_1_01 [F67FF5BE0D834E48919D8C9428108694] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:50,036 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784909823_0001_1_01, eventCount=1
2016-01-14 15:21:50,037 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:50,037 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-01-14 15:21:50,037 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: BA97675EAADF475DB321EDF1B0F7493F on vertex vertex_1452784909823_0001_1_00 [AD2E9C268989402CBB4848BC6BA7DFAF]
2016-01-14 15:21:50,037 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F67FF5BE0D834E48919D8C9428108694, vertexId=vertex_1452784909823_0001_1_01, initRequestedTime=1452784910010, initedTime=1452784910036, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:21:50,038 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784909823_0001_1_01 [F67FF5BE0D834E48919D8C9428108694] with distanceFromRoot: 0
2016-01-14 15:21:50,038 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784909823_0001_1_01 [F67FF5BE0D834E48919D8C9428108694] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:21:50,039 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784909823_0001_1_00 [AD2E9C268989402CBB4848BC6BA7DFAF] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:50,040 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {BA97675EAADF475DB321EDF1B0F7493F=forAllWorkUnits=true, update=[1]}
2016-01-14 15:21:50,040 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452784909823_0001_1_00 [AD2E9C268989402CBB4848BC6BA7DFAF] parallelism set to 1
2016-01-14 15:21:50,041 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784909823_0001_1_00 [AD2E9C268989402CBB4848BC6BA7DFAF] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:50,042 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784909823_0001_1_00, eventCount=1
2016-01-14 15:21:50,043 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AD2E9C268989402CBB4848BC6BA7DFAF, vertexId=vertex_1452784909823_0001_1_00, initRequestedTime=1452784910002, initedTime=1452784910042, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:21:50,043 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784909823_0001_1_00 [AD2E9C268989402CBB4848BC6BA7DFAF] with distanceFromRoot: 0
2016-01-14 15:21:50,043 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:50,043 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784909823_0001_1_00 [AD2E9C268989402CBB4848BC6BA7DFAF] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:21:50,044 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F67FF5BE0D834E48919D8C9428108694
2016-01-14 15:21:50,044 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784909823_0001_1_01 [F67FF5BE0D834E48919D8C9428108694]
2016-01-14 15:21:50,044 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:50,045 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784909823_0001_1_01, startRequestedTime=1452784910044, startedTime=1452784910044
2016-01-14 15:21:50,045 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784909823_0001_1_01 [F67FF5BE0D834E48919D8C9428108694] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:50,045 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in AD2E9C268989402CBB4848BC6BA7DFAF
2016-01-14 15:21:50,045 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784909823_0001_1_00 [AD2E9C268989402CBB4848BC6BA7DFAF]
2016-01-14 15:21:50,046 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:50,046 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784909823_0001_1_00, startRequestedTime=1452784910045, startedTime=1452784910045
2016-01-14 15:21:50,047 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784909823_0001_1_00 [AD2E9C268989402CBB4848BC6BA7DFAF] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:50,047 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:50,048 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:TASK_STARTED]: vertexName=F67FF5BE0D834E48919D8C9428108694, taskId=task_1452784909823_0001_1_01_000000, scheduledTime=1452784910047, launchTime=1452784910047
2016-01-14 15:21:50,048 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784909823_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:50,048 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452784909823_0001_1_01 for vertex: vertex_1452784909823_0001_1_02 [BA93E953F8744BC09E649F1F4D7F9C24] numStartedSources: 1 numSources: 2
2016-01-14 15:21:50,048 INFO impl.VertexImpl (VertexImpl.java:transition(3160)) - Cannot start vertex: vertex_1452784909823_0001_1_02 [BA93E953F8744BC09E649F1F4D7F9C24] numStartedSources: 1 numSources: 2
2016-01-14 15:21:50,049 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:50,049 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:TASK_STARTED]: vertexName=AD2E9C268989402CBB4848BC6BA7DFAF, taskId=task_1452784909823_0001_1_00_000000, scheduledTime=1452784910049, launchTime=1452784910049
2016-01-14 15:21:50,049 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784909823_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:50,050 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452784909823_0001_1_00 for vertex: vertex_1452784909823_0001_1_02 [BA93E953F8744BC09E649F1F4D7F9C24] numStartedSources: 2 numSources: 2
2016-01-14 15:21:50,050 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784909823_0001_1_02 [BA93E953F8744BC09E649F1F4D7F9C24] with distanceFromRoot: 1
2016-01-14 15:21:50,050 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784909823_0001_1_01 [F67FF5BE0D834E48919D8C9428108694]
2016-01-14 15:21:50,051 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784909823_0001_1_00 [AD2E9C268989402CBB4848BC6BA7DFAF]
2016-01-14 15:21:50,051 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F67FF5BE0D834E48919D8C9428108694 in vertex: BA93E953F8744BC09E649F1F4D7F9C24
2016-01-14 15:21:50,051 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: AD2E9C268989402CBB4848BC6BA7DFAF in vertex: BA93E953F8744BC09E649F1F4D7F9C24
2016-01-14 15:21:50,052 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: BA93E953F8744BC09E649F1F4D7F9C24 with 2 source tasks and 1 pending tasks
2016-01-14 15:21:50,052 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:50,053 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784909823_0001_1_02, startRequestedTime=1452784910051, startedTime=1452784910051
2016-01-14 15:21:50,053 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784909823_0001_1_02 [BA93E953F8744BC09E649F1F4D7F9C24] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:50,053 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8017674F884343859A63E3E0BA74A805/CD27E258D9E7405BA05B6426EE39338B] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: F67FF5BE0D834E48919D8C9428108694, VertexParallelism: 1, TaskAttemptID:attempt_1452784909823_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=47A2B96A5AD848EEBBCFA7CE07AED287, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BA93E953F8744BC09E649F1F4D7F9C24, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:21:50,054 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784909823_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:50,054 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8017674F884343859A63E3E0BA74A805/CD27E258D9E7405BA05B6426EE39338B] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: AD2E9C268989402CBB4848BC6BA7DFAF, VertexParallelism: 1, TaskAttemptID:attempt_1452784909823_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BA97675EAADF475DB321EDF1B0F7493F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BA93E953F8744BC09E649F1F4D7F9C24, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:21:50,055 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784909823_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:50,055 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:50,055 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:50,056 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:21:50,057 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:21:50,057 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:21:50,058 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784909823_0001_00_000001
2016-01-14 15:21:50,059 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909823_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:50,060 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784909823_0001_00_000002
2016-01-14 15:21:50,060 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909823_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:50,062 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784909823_0001_00_000001
2016-01-14 15:21:50,062 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:21:50,064 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784909823_0001_00_000002
2016-01-14 15:21:50,064 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:21:50,065 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:50,065 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909823_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:50,065 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784909823_0001_00_000001, launchTime=1452784910065
2016-01-14 15:21:50,066 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909823_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:50,066 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:50,066 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784909823_0001_00_000001
2016-01-14 15:21:50,066 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784909823_0001_00_000002, launchTime=1452784910066
2016-01-14 15:21:50,068 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784909823_0001_00_000002
2016-01-14 15:21:50,068 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784909823_0001_1_01_000000_0] to container: [container_1452784909823_0001_00_000001]
2016-01-14 15:21:50,071 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909823_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:50,071 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784909823_0001_1_00_000000_0] to container: [container_1452784909823_0001_00_000002]
2016-01-14 15:21:50,071 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909823_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:50,072 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784909823_0001_00_000002 given task: attempt_1452784909823_0001_1_00_000000_0
2016-01-14 15:21:50,072 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784909823_0001_00_000001 given task: attempt_1452784909823_0001_1_01_000000_0
2016-01-14 15:21:50,072 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784909823_0001_1_01_000000_0] started. Is using containerId: [container_1452784909823_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:21:50,073 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F67FF5BE0D834E48919D8C9428108694, taskAttemptId=attempt_1452784909823_0001_1_01_000000_0, startTime=1452784910072, containerId=container_1452784909823_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784909823_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:21:50,073 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:50,073 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784909823_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:50,074 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784909823_0001_1_00_000000_0] started. Is using containerId: [container_1452784909823_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:21:50,074 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784909823_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784909823_0001_1_00_000000_0
2016-01-14 15:21:50 Starting to run new task attempt: attempt_1452784909823_0001_1_00_000000_0
2016-01-14 15:21:50,074 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784909823_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784909823_0001_1_01_000000_0
2016-01-14 15:21:50,074 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:50,074 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AD2E9C268989402CBB4848BC6BA7DFAF, taskAttemptId=attempt_1452784909823_0001_1_00_000000_0, startTime=1452784910073, containerId=container_1452784909823_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784909823_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:21:50,075 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784909823_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:50,075 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784909823_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:50,076 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784909823_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:50,074 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784909823_0001_1_00_000000_0
2016-01-14 15:21:50,076 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:50,076 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:50,076 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:50,076 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:50,076 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:50,076 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:50,077 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8017674F884343859A63E3E0BA74A805/CD27E258D9E7405BA05B6426EE39338B] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: AD2E9C268989402CBB4848BC6BA7DFAF, VertexParallelism: 1, TaskAttemptID:attempt_1452784909823_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BA97675EAADF475DB321EDF1B0F7493F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BA93E953F8744BC09E649F1F4D7F9C24, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:21:50,077 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:50 Starting to run new task attempt: attempt_1452784909823_0001_1_01_000000_0
2016-01-14 15:21:50,077 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784909823_0001_1_01_000000_0
2016-01-14 15:21:50,077 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:50,077 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:50,077 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:50,077 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:50,078 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:50,078 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:50,078 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8017674F884343859A63E3E0BA74A805/CD27E258D9E7405BA05B6426EE39338B] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: F67FF5BE0D834E48919D8C9428108694, VertexParallelism: 1, TaskAttemptID:attempt_1452784909823_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=47A2B96A5AD848EEBBCFA7CE07AED287, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BA93E953F8744BC09E649F1F4D7F9C24, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:21:50,078 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:50,082 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784909823_0001_1_00_000000_0
2016-01-14 15:21:50,083 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:50,083 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:50,083 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784909823_0001_1_01_000000_0
2016-01-14 15:21:50,083 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:50,083 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:50,088 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BA97675EAADF475DB321EDF1B0F7493F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:21:50,088 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=47A2B96A5AD848EEBBCFA7CE07AED287, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:21:50,089 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:50,089 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 47A2B96A5AD848EEBBCFA7CE07AED287
2016-01-14 15:21:50,089 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 47A2B96A5AD848EEBBCFA7CE07AED287
2016-01-14 15:21:50,089 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:50,090 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BA97675EAADF475DB321EDF1B0F7493F
2016-01-14 15:21:50,090 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: BA97675EAADF475DB321EDF1B0F7493F
2016-01-14 15:21:50,091 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:50,094 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784909823_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:21:50,094 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784909823_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:21:50,095 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:50,095 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:21:50,095 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 47A2B96A5AD848EEBBCFA7CE07AED287
2016-01-14 15:21:50,096 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 47A2B96A5AD848EEBBCFA7CE07AED287
2016-01-14 15:21:50,097 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=BA93E953F8744BC09E649F1F4D7F9C24, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:21:50,098 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:50,098 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: BA93E953F8744BC09E649F1F4D7F9C24
2016-01-14 15:21:50,100 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:21:50,100 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: BA93E953F8744BC09E649F1F4D7F9C24
2016-01-14 15:21:50,100 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:50,103 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:50,109 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F67FF5BE0D834E48919D8C9428108694, ordinal: 1
2016-01-14 15:21:50,109 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F67FF5BE0D834E48919D8C9428108694, mem on start (mb), free: 314, total: 516, max: 672
2016-01-14 15:21:50,113 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:50,113 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:50,114 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:50,115 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: BA93E953F8744BC09E649F1F4D7F9C24
2016-01-14 15:21:50,115 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:21:50,116 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: BA97675EAADF475DB321EDF1B0F7493F
2016-01-14 15:21:50,116 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BA97675EAADF475DB321EDF1B0F7493F
2016-01-14 15:21:50,117 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:50,117 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:50,118 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:21:50,118 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:50,119 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:21:50,119 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:21:50,120 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:21:50,120 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 47A2B96A5AD848EEBBCFA7CE07AED287, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:21:50,123 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, BA93E953F8744BC09E649F1F4D7F9C24, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:21:50,123 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:50,123 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 47A2B96A5AD848EEBBCFA7CE07AED287 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:50,124 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:21:50,124 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:50,121 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=BA93E953F8744BC09E649F1F4D7F9C24, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:21:50,124 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 47A2B96A5AD848EEBBCFA7CE07AED287
2016-01-14 15:21:50,126 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:50,126 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 47A2B96A5AD848EEBBCFA7CE07AED287
2016-01-14 15:21:50,125 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:50,126 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: BA93E953F8744BC09E649F1F4D7F9C24
2016-01-14 15:21:50,126 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:50,128 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:21:50,128 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: BA93E953F8744BC09E649F1F4D7F9C24
2016-01-14 15:21:50,130 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AD2E9C268989402CBB4848BC6BA7DFAF, ordinal: 0
2016-01-14 15:21:50,130 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AD2E9C268989402CBB4848BC6BA7DFAF, mem on start (mb), free: 314, total: 516, max: 672
2016-01-14 15:21:50,131 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:50,131 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:50,132 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:50,136 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: BA93E953F8744BC09E649F1F4D7F9C24
2016-01-14 15:21:50,136 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784909823_0001_1_01_000000_0
2016-01-14 15:21:50,137 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:50,137 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:50,137 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:21:50,137 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:50,137 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:21:50,138 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:21:50,138 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:21:50,138 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BA97675EAADF475DB321EDF1B0F7493F, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:21:50,138 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, BA93E953F8744BC09E649F1F4D7F9C24, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:21:50,138 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:50,138 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BA97675EAADF475DB321EDF1B0F7493F being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:50,138 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:21:50,139 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:50,136 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:21:50,138 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BA97675EAADF475DB321EDF1B0F7493F
2016-01-14 15:21:50,139 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:50,139 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BA97675EAADF475DB321EDF1B0F7493F
2016-01-14 15:21:50,140 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:50,140 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784909823_0001_1_00_000000_0
2016-01-14 15:21:50,140 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@43e83a14
2016-01-14 15:21:50,142 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-01-14 15:21:50,143 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt"] streamed: true, id: 47A2B96A5AD848EEBBCFA7CE07AED287
2016-01-14 15:21:50,143 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]], id: E529B087BC0346B89673EBF59DA5C282
2016-01-14 15:21:50,143 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@43e83a14
2016-01-14 15:21:50,143 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:21:50,143 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:21:50,144 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:21:50,143 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] E529B087BC0346B89673EBF59DA5C282
2016-01-14 15:21:50,145 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6f8dea32
2016-01-14 15:21:50,146 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@10c6db9a; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@6f7b6410; comparator=cascading.tuple.hadoop.util.TupleComparator@de68f9e; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:21:50,147 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:21:50,150 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:21:50,150 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6f8dea32
2016-01-14 15:21:50,151 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:21:50,151 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:21:50,150 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 15:21:50,151 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:21:50,152 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"] streamed: true, id: BA97675EAADF475DB321EDF1B0F7493F
2016-01-14 15:21:50,153 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]], id: E529B087BC0346B89673EBF59DA5C282
2016-01-14 15:21:50,153 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] E529B087BC0346B89673EBF59DA5C282
2016-01-14 15:21:50,153 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@4113ccfd; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@7af7176b; comparator=cascading.tuple.hadoop.util.TupleComparator@7422b0de; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:21:50,165 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:21:50,167 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:21:50,168 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:21:50,169 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:21:50,169 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:21:50,165 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:21:50,170 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:21:50,170 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:21:50,172 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-01-14 15:21:50,172 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:50,173 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F67FF5BE0D834E48919D8C9428108694, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:21:50,258 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:21:50,258 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:21:50,259 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:21:50,259 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:21:50,259 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:21:50,260 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 15:21:50,260 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:50,261 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AD2E9C268989402CBB4848BC6BA7DFAF, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:21:50,263 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784909823_0001_1_00_000000_0
2016-01-14 15:21:50,264 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AD2E9C268989402CBB4848BC6BA7DFAF, mem on close (mb), free: 111, total: 516, max: 672
2016-01-14 15:21:50,264 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:21:50,264 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784909823_0001_1_01_000000_0
2016-01-14 15:21:50,264 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:21:50,264 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-01-14 15:21:50,264 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-01-14 15:21:50,264 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F67FF5BE0D834E48919D8C9428108694, mem on close (mb), free: 111, total: 516, max: 672
2016-01-14 15:21:50,265 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:21:50,265 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:21:50,265 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-01-14 15:21:50,266 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-01-14 15:21:50,267 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:21:50,269 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:21:50 Completed running task attempt: attempt_1452784909823_0001_1_01_000000_0
2016-01-14 15:21:50 Completed running task attempt: attempt_1452784909823_0001_1_00_000000_0
2016-01-14 15:21:50,275 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784909823_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:50,276 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784909823_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:50,278 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784909823_0001_1_01, eventCount=1
2016-01-14 15:21:50,279 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:50,279 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F67FF5BE0D834E48919D8C9428108694, taskAttemptId=attempt_1452784909823_0001_1_01_000000_0, startTime=1452784910072, finishTime=1452784910279, timeTaken=207, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=92, FILE_BYTES_WRITTEN=254, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=541589504, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784910173, Process_Duration=91, Process_End_Time=1452784910264, Read_Duration=88, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 15:21:50,279 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784909823_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:50,279 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784909823_0001_1_01_000000_0
2016-01-14 15:21:50,280 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:50,280 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:TASK_FINISHED]: vertexName=F67FF5BE0D834E48919D8C9428108694, taskId=task_1452784909823_0001_1_01_000000, startTime=1452784910072, finishTime=1452784910280, timeTaken=208, status=SUCCEEDED, successfulAttemptID=attempt_1452784909823_0001_1_01_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=92, FILE_BYTES_WRITTEN=254, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=541589504, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784910173, Process_Duration=91, Process_End_Time=1452784910264, Read_Duration=88, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 15:21:50,280 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784909823_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:50,280 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784909823_0001_1_01 [F67FF5BE0D834E48919D8C9428108694] : 1
2016-01-14 15:21:50,280 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784909823_0001_1_01 [F67FF5BE0D834E48919D8C9428108694]
2016-01-14 15:21:50,283 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:50,284 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:50,285 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:VERTEX_FINISHED]: vertexName=F67FF5BE0D834E48919D8C9428108694, vertexId=vertex_1452784909823_0001_1_01, initRequestedTime=1452784910010, initedTime=1452784910036, startRequestedTime=1452784910044, startedTime=1452784910044, finishTime=1452784910280, timeTaken=236, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=92, FILE_BYTES_WRITTEN=254, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=541589504, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784910173, Process_Duration=91, Process_End_Time=1452784910264, Read_Duration=88, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1452784910072, firstTasksToStart=[ task_1452784909823_0001_1_01_000000 ], lastTaskFinishTime=1452784910279, lastTasksToFinish=[ task_1452784909823_0001_1_01_000000 ], minTaskDuration=207, maxTaskDuration=207, avgTaskDuration=207.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784909823_0001_1_01_000000 ], longestDurationTasks=[ task_1452784909823_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:50,285 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=254, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=541589504, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452784910261, Process_Duration=2, Process_End_Time=1452784910263, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-01-14 15:21:50,285 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784909823_0001_1_01 [F67FF5BE0D834E48919D8C9428108694] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:50,285 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452784909823_0001_1_02 [BA93E953F8744BC09E649F1F4D7F9C24] attempt: attempt_1452784909823_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:21:50,285 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909823_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:50,286 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=92, FILE_BYTES_WRITTEN=254, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=541589504, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452784910173, Process_Duration=91, Process_End_Time=1452784910264, Read_Duration=88, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-01-14 15:21:50,286 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784909823_0001_1_00, eventCount=1
2016-01-14 15:21:50,286 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:50,286 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:50,286 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:50,286 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:50,286 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784909823_0001_00_000001
2016-01-14 15:21:50,286 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784909823_0001_1_01 [F67FF5BE0D834E48919D8C9428108694] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 15:21:50,287 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:50,287 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AD2E9C268989402CBB4848BC6BA7DFAF, taskAttemptId=attempt_1452784909823_0001_1_00_000000_0, startTime=1452784910073, finishTime=1452784910287, timeTaken=214, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=254, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=541589504, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784910261, Process_Duration=2, Process_End_Time=1452784910263, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 15:21:50,287 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784909823_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:50,287 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784909823_0001_00_000001, stoppedTime=1452784910287, exitStatus=0
2016-01-14 15:21:50,287 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909823_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:50,288 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:50,288 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784909823_0001_00_000002
2016-01-14 15:21:50,287 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:50,288 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784909823_0001_1_00_000000_0
2016-01-14 15:21:50,289 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:50,289 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:TASK_FINISHED]: vertexName=AD2E9C268989402CBB4848BC6BA7DFAF, taskId=task_1452784909823_0001_1_00_000000, startTime=1452784910073, finishTime=1452784910288, timeTaken=215, status=SUCCEEDED, successfulAttemptID=attempt_1452784909823_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=254, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=541589504, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784910261, Process_Duration=2, Process_End_Time=1452784910263, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 15:21:50,289 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784909823_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:50,290 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452784909823_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:21:50,290 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784909823_0001_1_00 [AD2E9C268989402CBB4848BC6BA7DFAF] : 1
2016-01-14 15:21:50,290 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784909823_0001_1_00 [AD2E9C268989402CBB4848BC6BA7DFAF]
2016-01-14 15:21:50,291 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:50,291 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:VERTEX_FINISHED]: vertexName=AD2E9C268989402CBB4848BC6BA7DFAF, vertexId=vertex_1452784909823_0001_1_00, initRequestedTime=1452784910002, initedTime=1452784910042, startRequestedTime=1452784910045, startedTime=1452784910045, finishTime=1452784910290, timeTaken=245, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=254, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=541589504, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784910261, Process_Duration=2, Process_End_Time=1452784910263, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1452784910073, firstTasksToStart=[ task_1452784909823_0001_1_00_000000 ], lastTaskFinishTime=1452784910287, lastTasksToFinish=[ task_1452784909823_0001_1_00_000000 ], minTaskDuration=214, maxTaskDuration=214, avgTaskDuration=214.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784909823_0001_1_00_000000 ], longestDurationTasks=[ task_1452784909823_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:50,291 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784909823_0001_1_00 [AD2E9C268989402CBB4848BC6BA7DFAF] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:50,292 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909823_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:50,292 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784909823_0001_00_000002, stoppedTime=1452784910292, exitStatus=0
2016-01-14 15:21:50,292 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909823_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:50,292 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909823_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:21:50,293 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452784909823_0001_1_02 [BA93E953F8744BC09E649F1F4D7F9C24] attempt: attempt_1452784909823_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:21:50,293 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: BA93E953F8744BC09E649F1F4D7F9C24
2016-01-14 15:21:50,293 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784909823_0001_1_02 [BA93E953F8744BC09E649F1F4D7F9C24]
2016-01-14 15:21:50,294 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784909823_0001_1_00 [AD2E9C268989402CBB4848BC6BA7DFAF] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 15:21:50,292 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:50,295 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452784909823_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 15:21:50,295 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:50,295 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:TASK_STARTED]: vertexName=BA93E953F8744BC09E649F1F4D7F9C24, taskId=task_1452784909823_0001_1_02_000000, scheduledTime=1452784910295, launchTime=1452784910295
2016-01-14 15:21:50,296 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784909823_0001_1_02_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:50,296 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909823_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:21:50,296 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784909823_0001_1_02 [BA93E953F8744BC09E649F1F4D7F9C24]
2016-01-14 15:21:50,297 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8017674F884343859A63E3E0BA74A805/CD27E258D9E7405BA05B6426EE39338B] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: BA93E953F8744BC09E649F1F4D7F9C24, VertexParallelism: 1, TaskAttemptID:attempt_1452784909823_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AD2E9C268989402CBB4848BC6BA7DFAF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=F67FF5BE0D834E48919D8C9428108694, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=31C2DBAB314E413BB81EB5A2C43198DB, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:50,297 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784909823_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:21:50,297 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784909823_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:50,297 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784909823_0001_00_000001, Exiting
2016-01-14 15:21:50,297 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784909823_0001_00_000001
2016-01-14 15:21:50,297 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784909823_0001_00_000001
2016-01-14 15:21:50,297 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784909823_0001_1_02_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:50,298 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:50,298 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784909823_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:21:50,299 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784909823_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:50,298 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784909823_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:50,299 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784909823_0001_00_000002, Exiting
2016-01-14 15:21:50,299 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784909823_0001_00_000002
2016-01-14 15:21:50,299 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784909823_0001_00_000002
2016-01-14 15:21:50,299 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784909823_0001_00_000001 completed successfully
2016-01-14 15:21:50,300 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784909823_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:50,300 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784909823_0001_00_000003
2016-01-14 15:21:50,300 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909823_0001_00_000003 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:50,300 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784909823_0001_00_000002 completed successfully
2016-01-14 15:21:50,302 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784909823_0001_00_000003
2016-01-14 15:21:50,303 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 15:21:50,304 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909823_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:21:50,305 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909823_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:21:50,306 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909823_0001_00_000003 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:50,306 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:50,306 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784909823_0001_00_000003, launchTime=1452784910306
2016-01-14 15:21:50,307 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784909823_0001_00_000003
2016-01-14 15:21:50,308 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784909823_0001_1_02_000000_0] to container: [container_1452784909823_0001_00_000003]
2016-01-14 15:21:50,308 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909823_0001_00_000003 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:50,308 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784909823_0001_00_000003 given task: attempt_1452784909823_0001_1_02_000000_0
2016-01-14 15:21:50,309 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784909823_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784909823_0001_1_02_000000_0
2016-01-14 15:21:50 Starting to run new task attempt: attempt_1452784909823_0001_1_02_000000_0
2016-01-14 15:21:50,310 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784909823_0001_1_02_000000_0
2016-01-14 15:21:50,310 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:50,311 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:50,311 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:50,311 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:50,312 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:50,312 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:50,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8017674F884343859A63E3E0BA74A805/CD27E258D9E7405BA05B6426EE39338B] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: BA93E953F8744BC09E649F1F4D7F9C24, VertexParallelism: 1, TaskAttemptID:attempt_1452784909823_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AD2E9C268989402CBB4848BC6BA7DFAF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=F67FF5BE0D834E48919D8C9428108694, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=31C2DBAB314E413BB81EB5A2C43198DB, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:50,313 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:50,314 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784909823_0001_1_02_000000_0
2016-01-14 15:21:50,314 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:50,314 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:50,316 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=AD2E9C268989402CBB4848BC6BA7DFAF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:21:50,317 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784909823_0001_1_02_000000_0] started. Is using containerId: [container_1452784909823_0001_00_000003] on NM: [127.0.0.1:0]
2016-01-14 15:21:50,317 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:50,317 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:50,318 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: AD2E9C268989402CBB4848BC6BA7DFAF
2016-01-14 15:21:50,318 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BA93E953F8744BC09E649F1F4D7F9C24, taskAttemptId=attempt_1452784909823_0001_1_02_000000_0, startTime=1452784910309, containerId=container_1452784909823_0001_00_000003, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784909823_0001_00_000003/teamcity, completedLogs=
2016-01-14 15:21:50,319 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784909823_0001_1_02_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:50,319 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784909823_0001_1_02_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:50,318 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F67FF5BE0D834E48919D8C9428108694, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:21:50,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:50,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F67FF5BE0D834E48919D8C9428108694
2016-01-14 15:21:50,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=31C2DBAB314E413BB81EB5A2C43198DB, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:21:50,396 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:50,396 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 31C2DBAB314E413BB81EB5A2C43198DB
2016-01-14 15:21:50,396 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:21:50,396 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 31C2DBAB314E413BB81EB5A2C43198DB
2016-01-14 15:21:50,396 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:21:50,396 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:21:50,397 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: F67FF5BE0D834E48919D8C9428108694
2016-01-14 15:21:50,319 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:50,395 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:21:50,398 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:21:50,398 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: AD2E9C268989402CBB4848BC6BA7DFAF
2016-01-14 15:21:50,395 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784909823_0001_1_02_000000_0 sent events: (0-2)
2016-01-14 15:21:50,398 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F67FF5BE0D834E48919D8C9428108694
2016-01-14 15:21:50,401 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:50,406 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: AD2E9C268989402CBB4848BC6BA7DFAF
2016-01-14 15:21:50,415 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BA93E953F8744BC09E649F1F4D7F9C24, ordinal: 2
2016-01-14 15:21:50,415 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BA93E953F8744BC09E649F1F4D7F9C24, mem on start (mb), free: 559, total: 589, max: 672
2016-01-14 15:21:50,415 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:50,416 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 3 initializers to finish
2016-01-14 15:21:50,416 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:50,416 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:50,416 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:21:50,416 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 31C2DBAB314E413BB81EB5A2C43198DB
2016-01-14 15:21:50,416 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:50,416 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:50,417 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:21:50,417 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:50,417 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:21:50,417 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:21:50,417 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345
2016-01-14 15:21:50,418 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 1268357504, TotalRequestedScaled: 6.0881160192E8, TotalJVMHeap: 704643072, TotalAvailable: 461541212, TotalRequested/TotalJVMHeap:1.80
2016-01-14 15:21:50,418 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 31C2DBAB314E413BB81EB5A2C43198DB, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:21:50,418 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F67FF5BE0D834E48919D8C9428108694, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-01-14 15:21:50,418 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, AD2E9C268989402CBB4848BC6BA7DFAF, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-01-14 15:21:50,418 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:50,418 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: AD2E9C268989402CBB4848BC6BA7DFAF
2016-01-14 15:21:50,418 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: AD2E9C268989402CBB4848BC6BA7DFAF being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:50,419 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F67FF5BE0D834E48919D8C9428108694
2016-01-14 15:21:50,419 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F67FF5BE0D834E48919D8C9428108694 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:50,419 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 2
2016-01-14 15:21:50,420 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 2 IOs to start
2016-01-14 15:21:50,421 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:21:50,422 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:21:50,422 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: AD2E9C268989402CBB4848BC6BA7DFAF with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:21:50,422 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-01-14 15:21:50,422 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:21:50,422 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:21:50,422 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:21:50,423 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:21:50,425 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F67FF5BE0D834E48919D8C9428108694 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:21:50,425 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-01-14 15:21:50,425 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:21:50,427 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:21:50,428 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: AD2E9C268989402CBB4848BC6BA7DFAF
2016-01-14 15:21:50,428 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:50,428 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F67FF5BE0D834E48919D8C9428108694
2016-01-14 15:21:50,429 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:50,430 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784909823_0001_1_02_000000_0
2016-01-14 15:21:50,430 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452784909823_0001_1_01_000000_0_10057, runDuration: 172103, ]
2016-01-14 15:21:50,431 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452784909823_0001_1_00_000000_0_10058, runDuration: 146747, ]
2016-01-14 15:21:50,432 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452784909823_0001_1_00_000000_0_10058] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1452784910432, TimeTaken=1, Rate=0.09 MB/s
2016-01-14 15:21:50,432 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : AD2E9C268989402CBB4848BC6BA7DFAF
2016-01-14 15:21:50,433 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:21:50,433 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [AD2E9C268989402CBB4848BC6BA7DFAF] #9 in 2ms
2016-01-14 15:21:50,433 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [AD2E9C268989402CBB4848BC6BA7DFAF] #9, status:false, isInterrupted:false
2016-01-14 15:21:50,434 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]]
2016-01-14 15:21:50,434 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] streamed: true, id: E529B087BC0346B89673EBF59DA5C282
2016-01-14 15:21:50,435 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:21:50,435 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:21:50,435 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup"], id: 31C2DBAB314E413BB81EB5A2C43198DB
2016-01-14 15:21:50,435 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-01-14 15:21:50,436 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:21:50,436 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:21:50,435 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:21:50,436 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-01-14 15:21:50,436 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : AD2E9C268989402CBB4848BC6BA7DFAF
2016-01-14 15:21:50,436 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:21:50,436 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup"]
2016-01-14 15:21:50,438 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452784909823_0001_1_01_000000_0_10057] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1452784910438, TimeTaken=5, Rate=0.02 MB/s
2016-01-14 15:21:50,438 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F67FF5BE0D834E48919D8C9428108694
2016-01-14 15:21:50,439 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:21:50,439 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [F67FF5BE0D834E48919D8C9428108694] #10 in 6ms
2016-01-14 15:21:50,439 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] E529B087BC0346B89673EBF59DA5C282, for 2 inputs
2016-01-14 15:21:50,439 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] E529B087BC0346B89673EBF59DA5C282, for 2 inputs
2016-01-14 15:21:50,440 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F67FF5BE0D834E48919D8C9428108694] #10, status:false, isInterrupted:false
2016-01-14 15:21:50,442 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(89)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2016-01-14 15:21:50,442 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:21:50,443 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-01-14 15:21:50,443 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:21:50,443 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:21:50,443 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-01-14 15:21:50,443 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F67FF5BE0D834E48919D8C9428108694
2016-01-14 15:21:50,443 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:21:50,444 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(94)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2016-01-14 15:21:50,444 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BA93E953F8744BC09E649F1F4D7F9C24, all 2 inputs ready in: 00:00:00.000
2016-01-14 15:21:50,445 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@342236a9; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3232abfc
2016-01-14 15:21:50,445 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@1cb94723; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@47ed5990
2016-01-14 15:21:50,448 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:21:50,448 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:21:50,449 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452784909823_0001_1_02_000000_0
2016-01-14 15:21:50,450 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452784909823_0001_1_02_000000_0 given a go for committing the task output.
2016-01-14 15:21:50,451 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784909823_0001_1_02_000000_0
2016-01-14 15:21:50,451 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BA93E953F8744BC09E649F1F4D7F9C24, mem on close (mb), free: 555, total: 589, max: 672
2016-01-14 15:21:50,452 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: AD2E9C268989402CBB4848BC6BA7DFAF
2016-01-14 15:21:50,452 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:21:50,453 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:21:50,453 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:21:50,453 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:21:50,453 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:21:50,454 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F67FF5BE0D834E48919D8C9428108694
2016-01-14 15:21:50,455 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:21:50,456 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:21:50,456 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:21:50,456 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:21:50,457 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:21:50 Completed running task attempt: attempt_1452784909823_0001_1_02_000000_0
2016-01-14 15:21:50,458 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784909823_0001_1_02_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:50,460 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:50,460 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BA93E953F8744BC09E649F1F4D7F9C24, taskAttemptId=attempt_1452784909823_0001_1_02_000000_0, startTime=1452784910309, finishTime=1452784910459, timeTaken=150, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=572, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=74, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=618135552, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=20, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784910444, Process_Duration=7, Process_End_Time=1452784910451, Read_Duration=2, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-01-14 15:21:50,460 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784909823_0001_1_02_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:50,460 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784909823_0001_1_02_000000_0
2016-01-14 15:21:50,460 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:50,461 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:50,461 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:TASK_FINISHED]: vertexName=BA93E953F8744BC09E649F1F4D7F9C24, taskId=task_1452784909823_0001_1_02_000000, startTime=1452784910309, finishTime=1452784910460, timeTaken=151, status=SUCCEEDED, successfulAttemptID=attempt_1452784909823_0001_1_02_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=572, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=74, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=618135552, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=20, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784910444, Process_Duration=7, Process_End_Time=1452784910451, Read_Duration=2, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-01-14 15:21:50,461 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784909823_0001_1_02_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:50,461 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784909823_0001_1_02 [BA93E953F8744BC09E649F1F4D7F9C24] : 1
2016-01-14 15:21:50,461 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784909823_0001_1_02 [BA93E953F8744BC09E649F1F4D7F9C24]
2016-01-14 15:21:50,462 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:50,463 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:VERTEX_FINISHED]: vertexName=BA93E953F8744BC09E649F1F4D7F9C24, vertexId=vertex_1452784909823_0001_1_02, initRequestedTime=1452784910017, initedTime=1452784910033, startRequestedTime=1452784910051, startedTime=1452784910051, finishTime=1452784910461, timeTaken=410, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=572, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=74, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=618135552, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=20, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784910444, Process_Duration=7, Process_End_Time=1452784910451, Read_Duration=2, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1452784910309, firstTasksToStart=[ task_1452784909823_0001_1_02_000000 ], lastTaskFinishTime=1452784910459, lastTasksToFinish=[ task_1452784909823_0001_1_02_000000 ], minTaskDuration=150, maxTaskDuration=150, avgTaskDuration=150.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784909823_0001_1_02_000000 ], longestDurationTasks=[ task_1452784909823_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:50,463 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784909823_0001_1_02 [BA93E953F8744BC09E649F1F4D7F9C24] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:50,464 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909823_0001_00_000003 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:50,464 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784909823_0001_00_000003, stoppedTime=1452784910464, exitStatus=0
2016-01-14 15:21:50,464 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:50,464 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909823_0001_00_000003 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:50,465 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784909823_0001_1_02 [BA93E953F8744BC09E649F1F4D7F9C24] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 15:21:50,465 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452784909823_0001_1
2016-01-14 15:21:50,465 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452784909823_0001_1
2016-01-14 15:21:50,465 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 31C2DBAB314E413BB81EB5A2C43198DB for vertex: vertex_1452784909823_0001_1_02 [BA93E953F8744BC09E649F1F4D7F9C24]
2016-01-14 15:21:50,471 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=572, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=74, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=618135552, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=20, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=9][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452784910444, Process_Duration=7, Process_End_Time=1452784910451, Read_Duration=2, Tuples_Read=10, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-01-14 15:21:50,471 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:50,471 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:50,471 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784909823_0001_00_000003
2016-01-14 15:21:50,472 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784909823_0001_00_000003 is valid, but no longer registered, and will be killed
2016-01-14 15:21:50,472 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784909823_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:50,472 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784909823_0001_00_000003, Exiting
2016-01-14 15:21:50,472 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784909823_0001_00_000003
2016-01-14 15:21:50,472 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784909823_0001_00_000003
2016-01-14 15:21:50,474 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784909823_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:50,475 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784909823_0001_00_000003 completed successfully
2016-01-14 15:21:50,475 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452784909823_0001_1_00 [AD2E9C268989402CBB4848BC6BA7DFAF]
2016-01-14 15:21:50,475 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452784909823_0001_1_01 [F67FF5BE0D834E48919D8C9428108694]
2016-01-14 15:21:50,478 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452784909823_0001_1, queueSize=0
2016-01-14 15:21:50,478 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:21:50,478 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784909823_0001_1][Event:DAG_FINISHED]: dagId=dag_1452784909823_0001_1, startTime=1452784910001, finishTime=1452784910475, timeTaken=474, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=2, File System Counters, FILE_BYTES_READ=470, FILE_BYTES_WRITTEN=1080, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=74, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1701314560, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=15, OUTPUT_BYTES=150, OUTPUT_BYTES_WITH_OVERHEAD=182, OUTPUT_BYTES_PHYSICAL=190, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=20, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4358354730878, Process_Duration=100, Process_End_Time=4358354730978, Read_Duration=92, Tuples_Read=20, Tuples_Written=15, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=5
2016-01-14 15:21:50,478 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452784909823_0001_1 finished with state: SUCCEEDED
2016-01-14 15:21:50,478 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784909823_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:21:50,478 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452784909823_0001_00_000003
2016-01-14 15:21:50,479 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784909823_0001_00_000003 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:21:50 Completed Dag: dag_1452784909823_0001_1
2016-01-14 15:21:50,479 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452784909823_0001_1, dagState=SUCCEEDED
2016-01-14 15:21:50,480 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784909823_0001_1_post
2016-01-14 15:21:50,480 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:50,480 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:21:50,486 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...estCoGroup/complexcogroup, applicationId=application_1452784909823_0001
2016-01-14 15:21:50,487 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:21:50,487 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:21:50,488 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:21:50,488 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:21:50,488 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:21:50,488 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:21:50,489 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:21:50,489 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:21:50,489 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:21:50,489 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:21:50,490 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:21:50,490 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:21:50,490 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:21:50,495 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8017674F884343859A63E3E0BA74A805/CD27E258D9E7405BA05B6426EE39338B/.tez/application_1452784909823_0001
2016-01-14 15:21:50,497 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] completed in: 00:00.692
2016-01-14 15:21:50,568 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup
2016-01-14 15:21:50,590 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:21:50,590 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:21:50,623 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-01-14 15:21:50,630 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-01-14 15:21:50,630 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 15:21:50,630 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 15:21:50,631 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:21:50,652 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-01-14 15:21:50,653 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace
2016-01-14 15:21:50,662 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 15:21:50,662 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 15:21:50,663 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace"]
2016-01-14 15:21:50,663 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 15:21:50,663 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 15:21:50,663 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 15:21:50,664 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...tform/testReplace/replace
2016-01-14 15:21:50,664 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:21:50,682 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 15:21:50,687 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:50,687 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:21:50,688 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:21:50,697 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/6DCBC68402D14972808AD713BF385046/B13259E6701E479284459E447B9249EA/.tez/application_1452784910687_0001 doesn't exist and is created
2016-01-14 15:21:50,712 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:21:50,712 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:21:50,712 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/6DCBC68402D14972808AD713BF385046/B13259E6701E479284459E447B9249EA/.tez/application_1452784910687_0001
2016-01-14 15:21:50,715 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452784910687_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:50,724 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:21:50,725 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:21:50,726 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:21:50,727 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:21:50,727 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/6DCBC68402D14972808AD713BF385046/B13259E6701E479284459E447B9249EA/.tez/application_1452784910687_0001/localmode-log-dir
2016-01-14 15:21:50,727 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/6DCBC68402D14972808AD713BF385046/B13259E6701E479284459E447B9249EA/.tez/application_1452784910687_0001/localmode-log-dir/history.txt.appattempt_1452784910687_0001_000000, maxErrors=10
2016-01-14 15:21:50,727 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:21:50,727 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452784910687_0001_000000, appSubmitTime=1452784910714, launchTime=1452784910715
2016-01-14 15:21:50,731 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:21:50,732 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:21:50,733 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:21:50,735 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:43062
2016-01-14 15:21:50,746 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:21:50,747 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:21:50,749 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452784910687_0001_000000, startTime=1452784910749
2016-01-14 15:21:50,749 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:21:50,749 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:21:50,812 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:21:50,813 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:21:50,813 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...tform/testReplace/replace, applicationId=application_1452784910687_0001, dagName=[6DCBC68402D14972808AD713BF385046/B13259E6701E479284459E447B9249EA] test/(1/1) ...tform/testReplace/replace
2016-01-14 15:21:50,828 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [6DCBC68402D14972808AD713BF385046/B13259E6701E479284459E447B9249EA] test/(1/1) ...tform/testReplace/replace
2016-01-14 15:21:50,830 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452784910687_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/6DCBC68402D14972808AD713BF385046/B13259E6701E479284459E447B9249EA/.tez/application_1452784910687_0001/localmode-log-dir/dag_1452784910687_0001_1.dot
2016-01-14 15:21:50,831 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/6DCBC68402D14972808AD713BF385046/B13259E6701E479284459E447B9249EA/.tez/application_1452784910687_0001/localmode-log-dir/dag_1452784910687_0001_1-tez-dag.pb.txt
2016-01-14 15:21:50,838 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784910687_0001_1
2016-01-14 15:21:50,838 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:50,839 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [6DCBC68402D14972808AD713BF385046/B13259E6701E479284459E447B9249EA] test/(1/1) ...tform/testReplace/replace
2016-01-14 15:21:50 Running Dag: dag_1452784910687_0001_1
2016-01-14 15:21:50,851 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:21:50,852 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784910687_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452784910687_0001_1, submitTime=1452784910829
2016-01-14 15:21:50,852 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B14FC93AC55747BB80004C72B036C5EA
2016-01-14 15:21:50,852 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:21:50,853 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:21:50,853 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784910687_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452784910687_0001_1, initTime=1452784910852
2016-01-14 15:21:50,853 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784910687_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:21:50,854 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784910687_0001_1][Event:DAG_STARTED]: dagID=dag_1452784910687_0001_1, startTime=1452784910853
2016-01-14 15:21:50,854 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:21:50,854 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:21:50,855 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784910687_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:21:50,855 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B14FC93AC55747BB80004C72B036C5EA : {F497DBFE1B5A44F794920A907D543563={InputName=F497DBFE1B5A44F794920A907D543563}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:21:50,855 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: F497DBFE1B5A44F794920A907D543563, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:21:50,855 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452784910687_0001_1_00 [B14FC93AC55747BB80004C72B036C5EA]
2016-01-14 15:21:50,855 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452784910687_0001_1_00 [B14FC93AC55747BB80004C72B036C5EA]
2016-01-14 15:21:50,855 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452784910687_0001_1_00 [B14FC93AC55747BB80004C72B036C5EA]
2016-01-14 15:21:50,855 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...tform/testReplace/replace, applicationId=application_1452784910687_0001, dagName=[6DCBC68402D14972808AD713BF385046/B13259E6701E479284459E447B9249EA] test/(1/1) ...tform/testReplace/replace
2016-01-14 15:21:50,855 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452784910687_0001_1_00 [B14FC93AC55747BB80004C72B036C5EA]. Starting root input initializers: 1
2016-01-14 15:21:50,856 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784910687_0001_1_00 [B14FC93AC55747BB80004C72B036C5EA] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:21:50,857 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: F497DBFE1B5A44F794920A907D543563 on vertex vertex_1452784910687_0001_1_00 [B14FC93AC55747BB80004C72B036C5EA]
2016-01-14 15:21:50,865 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:21:50,867 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input F497DBFE1B5A44F794920A907D543563 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:21:50,867 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:21:50,869 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:21:50,871 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-01-14 15:21:50,872 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452784910687_0001, with dag id: dag_1452784910687_0001_1
2016-01-14 15:21:50,872 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: F497DBFE1B5A44F794920A907D543563 on vertex vertex_1452784910687_0001_1_00 [B14FC93AC55747BB80004C72B036C5EA]
2016-01-14 15:21:50,873 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784910687_0001_1_00 [B14FC93AC55747BB80004C72B036C5EA] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:50,873 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {F497DBFE1B5A44F794920A907D543563=forAllWorkUnits=true, update=[1]}
2016-01-14 15:21:50,873 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452784910687_0001_1_00 [B14FC93AC55747BB80004C72B036C5EA] parallelism set to 1
2016-01-14 15:21:50,873 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784910687_0001_1_00 [B14FC93AC55747BB80004C72B036C5EA] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:50,874 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784910687_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784910687_0001_1_00, eventCount=1
2016-01-14 15:21:50,874 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452784910687_0001_1_00 [B14FC93AC55747BB80004C72B036C5EA]
2016-01-14 15:21:50,874 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=AD02903527C24C21BCAE980B22648573, vertexId=vertex_1452784910687_0001_1_00 [B14FC93AC55747BB80004C72B036C5EA], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:21:50,874 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=AD02903527C24C21BCAE980B22648573, vertexId=vertex_1452784910687_0001_1_00 [B14FC93AC55747BB80004C72B036C5EA]
2016-01-14 15:21:50,878 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=AD02903527C24C21BCAE980B22648573, vertexName=B14FC93AC55747BB80004C72B036C5EA, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:21:50,878 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=AD02903527C24C21BCAE980B22648573, vertexId=vertex_1452784910687_0001_1_00 [B14FC93AC55747BB80004C72B036C5EA]
2016-01-14 15:21:50,879 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:50,879 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784910687_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B14FC93AC55747BB80004C72B036C5EA, vertexId=vertex_1452784910687_0001_1_00, initRequestedTime=1452784910855, initedTime=1452784910879, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:21:50,880 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784910687_0001_1_00 [B14FC93AC55747BB80004C72B036C5EA] with distanceFromRoot: 0
2016-01-14 15:21:50,880 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784910687_0001_1_00 [B14FC93AC55747BB80004C72B036C5EA] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:21:50,881 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B14FC93AC55747BB80004C72B036C5EA
2016-01-14 15:21:50,881 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784910687_0001_1_00 [B14FC93AC55747BB80004C72B036C5EA]
2016-01-14 15:21:50,881 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:50,882 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784910687_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784910687_0001_1_00, startRequestedTime=1452784910881, startedTime=1452784910881
2016-01-14 15:21:50,882 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784910687_0001_1_00 [B14FC93AC55747BB80004C72B036C5EA] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:50,883 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:50,884 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784910687_0001_1][Event:TASK_STARTED]: vertexName=B14FC93AC55747BB80004C72B036C5EA, taskId=task_1452784910687_0001_1_00_000000, scheduledTime=1452784910883, launchTime=1452784910883
2016-01-14 15:21:50,884 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784910687_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:50,885 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784910687_0001_1_00 [B14FC93AC55747BB80004C72B036C5EA]
2016-01-14 15:21:50,885 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [6DCBC68402D14972808AD713BF385046/B13259E6701E479284459E447B9249EA] test/(1/1) ...tform/testReplace/replace, VertexName: B14FC93AC55747BB80004C72B036C5EA, VertexParallelism: 1, TaskAttemptID:attempt_1452784910687_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F497DBFE1B5A44F794920A907D543563, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AD02903527C24C21BCAE980B22648573, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:50,885 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784910687_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:50,886 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:50,886 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:21:50,886 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:21:50,887 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:21:50,887 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784910687_0001_00_000001
2016-01-14 15:21:50,887 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784910687_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:50,889 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784910687_0001_00_000001
2016-01-14 15:21:50,889 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:21:50,889 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784910687_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:50,889 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:50,890 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784910687_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784910687_0001_00_000001, launchTime=1452784910889
2016-01-14 15:21:50,890 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784910687_0001_00_000001
2016-01-14 15:21:50,891 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784910687_0001_1_00_000000_0] to container: [container_1452784910687_0001_00_000001]
2016-01-14 15:21:50,891 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784910687_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:50,892 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784910687_0001_00_000001 given task: attempt_1452784910687_0001_1_00_000000_0
2016-01-14 15:21:50,892 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784910687_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784910687_0001_1_00_000000_0
2016-01-14 15:21:50,892 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784910687_0001_1_00_000000_0] started. Is using containerId: [container_1452784910687_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:21:50,893 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:50,893 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784910687_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B14FC93AC55747BB80004C72B036C5EA, taskAttemptId=attempt_1452784910687_0001_1_00_000000_0, startTime=1452784910892, containerId=container_1452784910687_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784910687_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:21:50 Starting to run new task attempt: attempt_1452784910687_0001_1_00_000000_0
2016-01-14 15:21:50,893 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784910687_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:50,893 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784910687_0001_1_00_000000_0
2016-01-14 15:21:50,894 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:50,894 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:50,894 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:50,893 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784910687_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:50,895 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:50,895 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:50,895 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:50,896 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [6DCBC68402D14972808AD713BF385046/B13259E6701E479284459E447B9249EA] test/(1/1) ...tform/testReplace/replace, VertexName: B14FC93AC55747BB80004C72B036C5EA, VertexParallelism: 1, TaskAttemptID:attempt_1452784910687_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F497DBFE1B5A44F794920A907D543563, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AD02903527C24C21BCAE980B22648573, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:50,897 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:50,898 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784910687_0001_1_00_000000_0
2016-01-14 15:21:50,898 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:50,899 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:50,898 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784910687_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:21:50,900 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F497DBFE1B5A44F794920A907D543563, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:21:50,900 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=AD02903527C24C21BCAE980B22648573, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:21:50,900 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:50,901 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:50,900 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:50,901 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: AD02903527C24C21BCAE980B22648573
2016-01-14 15:21:50,902 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:21:50,902 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: AD02903527C24C21BCAE980B22648573
2016-01-14 15:21:50,901 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F497DBFE1B5A44F794920A907D543563
2016-01-14 15:21:50,903 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: F497DBFE1B5A44F794920A907D543563
2016-01-14 15:21:50,906 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:50,908 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:21:50,908 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: F497DBFE1B5A44F794920A907D543563
2016-01-14 15:21:50,908 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F497DBFE1B5A44F794920A907D543563
2016-01-14 15:21:50,913 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:21:50,913 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: AD02903527C24C21BCAE980B22648573
2016-01-14 15:21:50,913 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B14FC93AC55747BB80004C72B036C5EA, ordinal: 0
2016-01-14 15:21:50,913 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B14FC93AC55747BB80004C72B036C5EA, mem on start (mb), free: 485, total: 589, max: 672
2016-01-14 15:21:50,914 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:50,914 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:50,914 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:50,914 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:50,915 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:50,915 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:21:50,915 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:50,915 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:21:50,916 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:50,916 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:21:50,916 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-01-14 15:21:50,917 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, AD02903527C24C21BCAE980B22648573, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:21:50,917 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F497DBFE1B5A44F794920A907D543563, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:21:50,917 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:50,917 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F497DBFE1B5A44F794920A907D543563 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:50,918 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F497DBFE1B5A44F794920A907D543563
2016-01-14 15:21:50,918 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:50,918 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F497DBFE1B5A44F794920A907D543563
2016-01-14 15:21:50,918 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:21:50,919 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:50,919 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:50,919 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784910687_0001_1_00_000000_0
2016-01-14 15:21:50,920 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:21:50,921 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6faf2576
2016-01-14 15:21:50,922 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6faf2576
2016-01-14 15:21:50,922 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:21:50,922 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:21:50,923 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 15:21:50,924 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: F497DBFE1B5A44F794920A907D543563
2016-01-14 15:21:50,924 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace"], id: AD02903527C24C21BCAE980B22648573
2016-01-14 15:21:50,924 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace"]
2016-01-14 15:21:50,924 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 15:21:50,924 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:50,924 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B14FC93AC55747BB80004C72B036C5EA, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:21:50,926 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:21:50,926 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:21:50,927 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452784910687_0001_1_00_000000_0
2016-01-14 15:21:50,927 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452784910687_0001_1_00_000000_0 given a go for committing the task output.
2016-01-14 15:21:50,928 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784910687_0001_1_00_000000_0
2016-01-14 15:21:50,928 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B14FC93AC55747BB80004C72B036C5EA, mem on close (mb), free: 484, total: 589, max: 672
2016-01-14 15:21:50 Completed running task attempt: attempt_1452784910687_0001_1_00_000000_0
2016-01-14 15:21:50,929 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784910687_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:50,929 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=618135552, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10][cascading.flow.SliceCounters Process_Begin_Time=1452784910925, Process_Duration=3, Process_End_Time=1452784910928, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10, Tuples_Written=10]]
2016-01-14 15:21:50,930 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784910687_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B14FC93AC55747BB80004C72B036C5EA, taskAttemptId=attempt_1452784910687_0001_1_00_000000_0, startTime=1452784910892, finishTime=1452784910929, timeTaken=37, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=618135552, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452784910925, Process_Duration=3, Process_End_Time=1452784910928, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-01-14 15:21:50,930 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784910687_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:50,930 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784910687_0001_1_00_000000_0
2016-01-14 15:21:50,930 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:50,930 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:50,931 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:50,930 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:50,931 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784910687_0001_1][Event:TASK_FINISHED]: vertexName=B14FC93AC55747BB80004C72B036C5EA, taskId=task_1452784910687_0001_1_00_000000, startTime=1452784910892, finishTime=1452784910930, timeTaken=38, status=SUCCEEDED, successfulAttemptID=attempt_1452784910687_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=618135552, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452784910925, Process_Duration=3, Process_End_Time=1452784910928, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-01-14 15:21:50,931 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784910687_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:50,931 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784910687_0001_1_00 [B14FC93AC55747BB80004C72B036C5EA] : 1
2016-01-14 15:21:50,932 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784910687_0001_1_00 [B14FC93AC55747BB80004C72B036C5EA]
2016-01-14 15:21:50,932 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:50,932 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784910687_0001_00_000001
2016-01-14 15:21:50,933 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784910687_0001_1][Event:VERTEX_FINISHED]: vertexName=B14FC93AC55747BB80004C72B036C5EA, vertexId=vertex_1452784910687_0001_1_00, initRequestedTime=1452784910855, initedTime=1452784910879, startRequestedTime=1452784910881, startedTime=1452784910881, finishTime=1452784910932, timeTaken=51, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=618135552, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452784910925, Process_Duration=3, Process_End_Time=1452784910928, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10, vertexStats=firstTaskStartTime=1452784910892, firstTasksToStart=[ task_1452784910687_0001_1_00_000000 ], lastTaskFinishTime=1452784910929, lastTasksToFinish=[ task_1452784910687_0001_1_00_000000 ], minTaskDuration=37, maxTaskDuration=37, avgTaskDuration=37.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784910687_0001_1_00_000000 ], longestDurationTasks=[ task_1452784910687_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:50,933 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:50,933 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784910687_0001_1_00 [B14FC93AC55747BB80004C72B036C5EA] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:50,933 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784910687_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:50,934 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784910687_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784910687_0001_00_000001, stoppedTime=1452784910934, exitStatus=0
2016-01-14 15:21:50,934 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784910687_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:50,934 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784910687_0001_1_00 [B14FC93AC55747BB80004C72B036C5EA] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-01-14 15:21:50,934 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:50,934 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784910687_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:21:50,934 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452784910687_0001_1
2016-01-14 15:21:50,934 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784910687_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452784910687_0001_1
2016-01-14 15:21:50,935 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: AD02903527C24C21BCAE980B22648573 for vertex: vertex_1452784910687_0001_1_00 [B14FC93AC55747BB80004C72B036C5EA]
2016-01-14 15:21:50,939 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784910687_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:50,939 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784910687_0001_00_000001, Exiting
2016-01-14 15:21:50,939 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784910687_0001_00_000001
2016-01-14 15:21:50,940 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784910687_0001_00_000001
2016-01-14 15:21:50,941 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784910687_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:50,942 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784910687_0001_00_000001 completed successfully
2016-01-14 15:21:50,947 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452784910687_0001_1, queueSize=0
2016-01-14 15:21:50,947 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:21:50,947 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784910687_0001_1][Event:DAG_FINISHED]: dagId=dag_1452784910687_0001_1, startTime=1452784910853, finishTime=1452784910945, timeTaken=92, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=618135552, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452784910925, Process_Duration=3, Process_End_Time=1452784910928, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-01-14 15:21:50,947 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452784910687_0001_1 finished with state: SUCCEEDED
2016-01-14 15:21:50,947 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784910687_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:21:50,948 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784910687_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:21:50 Completed Dag: dag_1452784910687_0001_1
2016-01-14 15:21:50,948 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452784910687_0001_1, dagState=SUCCEEDED
2016-01-14 15:21:50,948 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784910687_0001_1_post
2016-01-14 15:21:50,948 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:50,949 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:21:50,949 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452784910687_0001_00_000001
2016-01-14 15:21:50,956 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...tform/testReplace/replace, applicationId=application_1452784910687_0001
2016-01-14 15:21:50,958 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:21:50,958 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:21:50,958 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:21:50,958 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:21:50,958 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:21:50,959 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:21:50,960 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:21:50,960 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:21:50,961 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:21:50,961 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:21:50,961 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:21:50,962 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:21:50,962 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:21:50,966 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/6DCBC68402D14972808AD713BF385046/B13259E6701E479284459E447B9249EA/.tez/application_1452784910687_0001
2016-01-14 15:21:50,967 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:00.305
2016-01-14 15:21:51,006 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testReplace/replace
2016-01-14 15:21:51,031 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:21:51,031 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:21:51,070 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-01-14 15:21:51,080 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.025
2016-01-14 15:21:51,081 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:21:51,081 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:21:51,081 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:21:51,096 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-01-14 15:21:51,104 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2
2016-01-14 15:21:51,152 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] starting
2016-01-14 15:21:51,152 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 15:21:51,153 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2"]
2016-01-14 15:21:51,153 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] parallel execution of steps is enabled: false
2016-01-14 15:21:51,153 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executing total steps: 1
2016-01-14 15:21:51,153 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] allocating management threads: 1
2016-01-14 15:21:51,155 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] starting step: (1/1) ...latform/testCount2/count2
2016-01-14 15:21:51,155 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:21:51,167 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tez session mode enabled: true
2016-01-14 15:21:51,172 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:51,172 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:21:51,172 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:21:51,182 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3F7415C5BAFF41899AFA67A1E60794F2/B1DA1752CEE8455BAAF7E5B479422B80/.tez/application_1452784911172_0001 doesn't exist and is created
2016-01-14 15:21:51,202 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:21:51,202 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:21:51,202 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3F7415C5BAFF41899AFA67A1E60794F2/B1DA1752CEE8455BAAF7E5B479422B80/.tez/application_1452784911172_0001
2016-01-14 15:21:51,205 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452784911172_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:51,216 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:21:51,217 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:21:51,218 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:21:51,218 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:21:51,219 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3F7415C5BAFF41899AFA67A1E60794F2/B1DA1752CEE8455BAAF7E5B479422B80/.tez/application_1452784911172_0001/localmode-log-dir
2016-01-14 15:21:51,219 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3F7415C5BAFF41899AFA67A1E60794F2/B1DA1752CEE8455BAAF7E5B479422B80/.tez/application_1452784911172_0001/localmode-log-dir/history.txt.appattempt_1452784911172_0001_000000, maxErrors=10
2016-01-14 15:21:51,219 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:21:51,219 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452784911172_0001_000000, appSubmitTime=1452784911205, launchTime=1452784911205
2016-01-14 15:21:51,226 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:21:51,227 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:21:51,227 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:21:51,227 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:21:51,231 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:35239
2016-01-14 15:21:51,233 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:21:51,234 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452784911172_0001_000000, startTime=1452784911234
2016-01-14 15:21:51,234 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:21:51,234 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:21:51,302 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:21:51,303 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:21:51,303 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testCount2/count2, applicationId=application_1452784911172_0001, dagName=[3F7415C5BAFF41899AFA67A1E60794F2/B1DA1752CEE8455BAAF7E5B479422B80] count/(1/1) ...latform/testCount2/count2
2016-01-14 15:21:51,322 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [3F7415C5BAFF41899AFA67A1E60794F2/B1DA1752CEE8455BAAF7E5B479422B80] count/(1/1) ...latform/testCount2/count2
2016-01-14 15:21:51,324 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452784911172_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3F7415C5BAFF41899AFA67A1E60794F2/B1DA1752CEE8455BAAF7E5B479422B80/.tez/application_1452784911172_0001/localmode-log-dir/dag_1452784911172_0001_1.dot
2016-01-14 15:21:51,324 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3F7415C5BAFF41899AFA67A1E60794F2/B1DA1752CEE8455BAAF7E5B479422B80/.tez/application_1452784911172_0001/localmode-log-dir/dag_1452784911172_0001_1-tez-dag.pb.txt
2016-01-14 15:21:51,331 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784911172_0001_1
2016-01-14 15:21:51,331 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:51,331 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [3F7415C5BAFF41899AFA67A1E60794F2/B1DA1752CEE8455BAAF7E5B479422B80] count/(1/1) ...latform/testCount2/count2
2016-01-14 15:21:51 Running Dag: dag_1452784911172_0001_1
2016-01-14 15:21:51,344 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:21:51,344 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911172_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452784911172_0001_1, submitTime=1452784911322
2016-01-14 15:21:51,344 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B82C2CB46B0F4F988BEDAF348077A2BD
2016-01-14 15:21:51,345 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:21:51,345 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911172_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452784911172_0001_1, initTime=1452784911344
2016-01-14 15:21:51,345 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784911172_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:21:51,345 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:21:51,346 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911172_0001_1][Event:DAG_STARTED]: dagID=dag_1452784911172_0001_1, startTime=1452784911345
2016-01-14 15:21:51,346 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:21:51,346 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784911172_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:21:51,347 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CCE6ECA6F6484DA1AEA1CC40AF6EC1DF : {C365F893389242AD960C7AA84D58C19E={InputName=C365F893389242AD960C7AA84D58C19E}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:21:51,347 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: C365F893389242AD960C7AA84D58C19E, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:21:51,347 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452784911172_0001_1_00 [CCE6ECA6F6484DA1AEA1CC40AF6EC1DF]
2016-01-14 15:21:51,347 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452784911172_0001_1_00 [CCE6ECA6F6484DA1AEA1CC40AF6EC1DF]
2016-01-14 15:21:51,347 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452784911172_0001_1_00 [CCE6ECA6F6484DA1AEA1CC40AF6EC1DF]
2016-01-14 15:21:51,347 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452784911172_0001_1_00 [CCE6ECA6F6484DA1AEA1CC40AF6EC1DF]. Starting root input initializers: 1
2016-01-14 15:21:51,346 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testCount2/count2, applicationId=application_1452784911172_0001, dagName=[3F7415C5BAFF41899AFA67A1E60794F2/B1DA1752CEE8455BAAF7E5B479422B80] count/(1/1) ...latform/testCount2/count2
2016-01-14 15:21:51,345 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:21:51,348 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:21:51,346 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:21:51,348 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784911172_0001_1_00 [CCE6ECA6F6484DA1AEA1CC40AF6EC1DF] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:21:51,348 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452784911172_0001_1_01 [B82C2CB46B0F4F988BEDAF348077A2BD]
2016-01-14 15:21:51,351 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: C365F893389242AD960C7AA84D58C19E on vertex vertex_1452784911172_0001_1_00 [CCE6ECA6F6484DA1AEA1CC40AF6EC1DF]
2016-01-14 15:21:51,351 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:21:51,352 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452784911172_0001_1_01 [B82C2CB46B0F4F988BEDAF348077A2BD]
2016-01-14 15:21:51,352 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452784911172_0001_1_01 [B82C2CB46B0F4F988BEDAF348077A2BD]
2016-01-14 15:21:51,352 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452784911172_0001_1_01 [B82C2CB46B0F4F988BEDAF348077A2BD]
2016-01-14 15:21:51,353 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=F790FA1E343B402598D4523147C8DDE4, vertexId=vertex_1452784911172_0001_1_01 [B82C2CB46B0F4F988BEDAF348077A2BD], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:21:51,353 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=F790FA1E343B402598D4523147C8DDE4, vertexId=vertex_1452784911172_0001_1_01 [B82C2CB46B0F4F988BEDAF348077A2BD]
2016-01-14 15:21:51,355 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:21:51,355 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=F790FA1E343B402598D4523147C8DDE4, vertexName=B82C2CB46B0F4F988BEDAF348077A2BD, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:21:51,355 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=F790FA1E343B402598D4523147C8DDE4, vertexId=vertex_1452784911172_0001_1_01 [B82C2CB46B0F4F988BEDAF348077A2BD]
2016-01-14 15:21:51,356 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input C365F893389242AD960C7AA84D58C19E asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:21:51,357 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:21:51,357 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:51,357 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:21:51,357 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911172_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B82C2CB46B0F4F988BEDAF348077A2BD, vertexId=vertex_1452784911172_0001_1_01, initRequestedTime=1452784911348, initedTime=1452784911357, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:21:51,358 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784911172_0001_1_01 [B82C2CB46B0F4F988BEDAF348077A2BD] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:21:51,360 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-01-14 15:21:51,360 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: C365F893389242AD960C7AA84D58C19E on vertex vertex_1452784911172_0001_1_00 [CCE6ECA6F6484DA1AEA1CC40AF6EC1DF]
2016-01-14 15:21:51,361 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784911172_0001_1_00 [CCE6ECA6F6484DA1AEA1CC40AF6EC1DF] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:51,361 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {C365F893389242AD960C7AA84D58C19E=forAllWorkUnits=true, update=[1]}
2016-01-14 15:21:51,362 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452784911172_0001_1_00 [CCE6ECA6F6484DA1AEA1CC40AF6EC1DF] parallelism set to 1
2016-01-14 15:21:51,362 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784911172_0001_1_00 [CCE6ECA6F6484DA1AEA1CC40AF6EC1DF] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:51,363 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911172_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784911172_0001_1_00, eventCount=1
2016-01-14 15:21:51,364 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911172_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CCE6ECA6F6484DA1AEA1CC40AF6EC1DF, vertexId=vertex_1452784911172_0001_1_00, initRequestedTime=1452784911347, initedTime=1452784911363, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:21:51,364 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784911172_0001_1_00 [CCE6ECA6F6484DA1AEA1CC40AF6EC1DF] with distanceFromRoot: 0
2016-01-14 15:21:51,364 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:51,364 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784911172_0001_1_00 [CCE6ECA6F6484DA1AEA1CC40AF6EC1DF] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:21:51,365 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CCE6ECA6F6484DA1AEA1CC40AF6EC1DF
2016-01-14 15:21:51,365 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784911172_0001_1_00 [CCE6ECA6F6484DA1AEA1CC40AF6EC1DF]
2016-01-14 15:21:51,366 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] submitted tez dag to app master: application_1452784911172_0001, with dag id: dag_1452784911172_0001_1
2016-01-14 15:21:51,367 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911172_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784911172_0001_1_00, startRequestedTime=1452784911365, startedTime=1452784911365
2016-01-14 15:21:51,367 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:51,367 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784911172_0001_1_00 [CCE6ECA6F6484DA1AEA1CC40AF6EC1DF] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:51,368 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:51,368 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911172_0001_1][Event:TASK_STARTED]: vertexName=CCE6ECA6F6484DA1AEA1CC40AF6EC1DF, taskId=task_1452784911172_0001_1_00_000000, scheduledTime=1452784911368, launchTime=1452784911368
2016-01-14 15:21:51,368 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784911172_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:51,370 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452784911172_0001_1_00 for vertex: vertex_1452784911172_0001_1_01 [B82C2CB46B0F4F988BEDAF348077A2BD] numStartedSources: 1 numSources: 1
2016-01-14 15:21:51,370 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784911172_0001_1_01 [B82C2CB46B0F4F988BEDAF348077A2BD] with distanceFromRoot: 1
2016-01-14 15:21:51,370 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784911172_0001_1_00 [CCE6ECA6F6484DA1AEA1CC40AF6EC1DF]
2016-01-14 15:21:51,371 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: CCE6ECA6F6484DA1AEA1CC40AF6EC1DF in vertex: B82C2CB46B0F4F988BEDAF348077A2BD
2016-01-14 15:21:51,371 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B82C2CB46B0F4F988BEDAF348077A2BD with 1 source tasks and 1 pending tasks
2016-01-14 15:21:51,371 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911172_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784911172_0001_1_01, startRequestedTime=1452784911371, startedTime=1452784911371
2016-01-14 15:21:51,371 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:51,372 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784911172_0001_1_01 [B82C2CB46B0F4F988BEDAF348077A2BD] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:51,374 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3F7415C5BAFF41899AFA67A1E60794F2/B1DA1752CEE8455BAAF7E5B479422B80] count/(1/1) ...latform/testCount2/count2, VertexName: CCE6ECA6F6484DA1AEA1CC40AF6EC1DF, VertexParallelism: 1, TaskAttemptID:attempt_1452784911172_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C365F893389242AD960C7AA84D58C19E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B82C2CB46B0F4F988BEDAF348077A2BD, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:21:51,375 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784911172_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:51,375 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:51,376 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:21:51,376 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:21:51,377 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:21:51,377 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784911172_0001_00_000001
2016-01-14 15:21:51,377 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784911172_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:51,379 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784911172_0001_00_000001
2016-01-14 15:21:51,380 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:21:51,380 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:51,380 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911172_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784911172_0001_00_000001, launchTime=1452784911380
2016-01-14 15:21:51,380 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784911172_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:51,381 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784911172_0001_00_000001
2016-01-14 15:21:51,382 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784911172_0001_1_00_000000_0] to container: [container_1452784911172_0001_00_000001]
2016-01-14 15:21:51,382 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784911172_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:51,382 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784911172_0001_00_000001 given task: attempt_1452784911172_0001_1_00_000000_0
2016-01-14 15:21:51,382 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784911172_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784911172_0001_1_00_000000_0
2016-01-14 15:21:51 Starting to run new task attempt: attempt_1452784911172_0001_1_00_000000_0
2016-01-14 15:21:51,383 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784911172_0001_1_00_000000_0
2016-01-14 15:21:51,383 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:51,383 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:51,383 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:51,383 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:51,383 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:51,383 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:51,384 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3F7415C5BAFF41899AFA67A1E60794F2/B1DA1752CEE8455BAAF7E5B479422B80] count/(1/1) ...latform/testCount2/count2, VertexName: CCE6ECA6F6484DA1AEA1CC40AF6EC1DF, VertexParallelism: 1, TaskAttemptID:attempt_1452784911172_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C365F893389242AD960C7AA84D58C19E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B82C2CB46B0F4F988BEDAF348077A2BD, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:21:51,384 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:51,384 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784911172_0001_1_00_000000_0
2016-01-14 15:21:51,384 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:51,384 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:51,385 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C365F893389242AD960C7AA84D58C19E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:21:51,386 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B82C2CB46B0F4F988BEDAF348077A2BD, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:21:51,386 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784911172_0001_1_00_000000_0] started. Is using containerId: [container_1452784911172_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:21:51,386 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:51,386 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B82C2CB46B0F4F988BEDAF348077A2BD
2016-01-14 15:21:51,385 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:51,385 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784911172_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:21:51,386 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:51,387 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C365F893389242AD960C7AA84D58C19E
2016-01-14 15:21:51,387 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: C365F893389242AD960C7AA84D58C19E
2016-01-14 15:21:51,390 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:21:51,390 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: C365F893389242AD960C7AA84D58C19E
2016-01-14 15:21:51,390 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C365F893389242AD960C7AA84D58C19E
2016-01-14 15:21:51,391 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:21:51,391 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B82C2CB46B0F4F988BEDAF348077A2BD
2016-01-14 15:21:51,391 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:51,391 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911172_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CCE6ECA6F6484DA1AEA1CC40AF6EC1DF, taskAttemptId=attempt_1452784911172_0001_1_00_000000_0, startTime=1452784911382, containerId=container_1452784911172_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784911172_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:21:51,391 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784911172_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:51,392 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784911172_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:51,393 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:51,398 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B82C2CB46B0F4F988BEDAF348077A2BD
2016-01-14 15:21:51,402 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CCE6ECA6F6484DA1AEA1CC40AF6EC1DF, ordinal: 0
2016-01-14 15:21:51,402 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CCE6ECA6F6484DA1AEA1CC40AF6EC1DF, mem on start (mb), free: 408, total: 512, max: 672
2016-01-14 15:21:51,403 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:51,403 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:51,403 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:51,404 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:51,404 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:51,404 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:21:51,404 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:51,404 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:21:51,405 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:21:51,405 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:21:51,405 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C365F893389242AD960C7AA84D58C19E, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:21:51,406 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B82C2CB46B0F4F988BEDAF348077A2BD, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:21:51,406 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:51,406 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C365F893389242AD960C7AA84D58C19E being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:51,406 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:21:51,407 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:51,406 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C365F893389242AD960C7AA84D58C19E
2016-01-14 15:21:51,408 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:51,408 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C365F893389242AD960C7AA84D58C19E
2016-01-14 15:21:51,408 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:51,409 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784911172_0001_1_00_000000_0
2016-01-14 15:21:51,409 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:21:51,410 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@440aa2fe
2016-01-14 15:21:51,411 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@440aa2fe
2016-01-14 15:21:51,412 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:21:51,412 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:21:51,412 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 15:21:51,413 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: C365F893389242AD960C7AA84D58C19E
2016-01-14 15:21:51,413 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: C272348216784898893E75C4A14A306F
2016-01-14 15:21:51,414 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] C272348216784898893E75C4A14A306F
2016-01-14 15:21:51,414 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@5e1011d8; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@7103ac26; comparator=cascading.tuple.hadoop.util.TupleComparator@281b2cc1; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:21:51,415 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:21:51,415 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:21:51,416 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:21:51,428 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:21:51,429 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:21:51,429 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:21:51,429 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:21:51,429 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:21:51,430 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 15:21:51,430 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:51,430 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CCE6ECA6F6484DA1AEA1CC40AF6EC1DF, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:21:51,433 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784911172_0001_1_00_000000_0
2016-01-14 15:21:51,434 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CCE6ECA6F6484DA1AEA1CC40AF6EC1DF, mem on close (mb), free: 306, total: 512, max: 672
2016-01-14 15:21:51,434 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:21:51,434 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:21:51,434 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2016-01-14 15:21:51,434 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 15:21:51,438 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:21:51 Completed running task attempt: attempt_1452784911172_0001_1_00_000000_0
2016-01-14 15:21:51,445 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784911172_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:51,446 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911172_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784911172_0001_1_00, eventCount=1
2016-01-14 15:21:51,446 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911172_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CCE6ECA6F6484DA1AEA1CC40AF6EC1DF, taskAttemptId=attempt_1452784911172_0001_1_00_000000_0, startTime=1452784911382, finishTime=1452784911446, timeTaken=64, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=536870912, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784911431, Process_Duration=2, Process_End_Time=1452784911433, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 15:21:51,446 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784911172_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:51,447 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:51,447 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784911172_0001_1_00_000000_0
2016-01-14 15:21:51,448 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911172_0001_1][Event:TASK_FINISHED]: vertexName=CCE6ECA6F6484DA1AEA1CC40AF6EC1DF, taskId=task_1452784911172_0001_1_00_000000, startTime=1452784911382, finishTime=1452784911447, timeTaken=65, status=SUCCEEDED, successfulAttemptID=attempt_1452784911172_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=536870912, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784911431, Process_Duration=2, Process_End_Time=1452784911433, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 15:21:51,448 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784911172_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:51,448 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784911172_0001_1_00 [CCE6ECA6F6484DA1AEA1CC40AF6EC1DF] : 1
2016-01-14 15:21:51,448 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784911172_0001_1_00 [CCE6ECA6F6484DA1AEA1CC40AF6EC1DF]
2016-01-14 15:21:51,448 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:51,449 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:51,450 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911172_0001_1][Event:VERTEX_FINISHED]: vertexName=CCE6ECA6F6484DA1AEA1CC40AF6EC1DF, vertexId=vertex_1452784911172_0001_1_00, initRequestedTime=1452784911347, initedTime=1452784911363, startRequestedTime=1452784911365, startedTime=1452784911365, finishTime=1452784911448, timeTaken=83, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=536870912, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784911431, Process_Duration=2, Process_End_Time=1452784911433, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452784911382, firstTasksToStart=[ task_1452784911172_0001_1_00_000000 ], lastTaskFinishTime=1452784911446, lastTasksToFinish=[ task_1452784911172_0001_1_00_000000 ], minTaskDuration=64, maxTaskDuration=64, avgTaskDuration=64.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784911172_0001_1_00_000000 ], longestDurationTasks=[ task_1452784911172_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:51,451 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784911172_0001_1_00 [CCE6ECA6F6484DA1AEA1CC40AF6EC1DF] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:51,451 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:51,452 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784911172_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:51,452 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911172_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784911172_0001_00_000001, stoppedTime=1452784911452, exitStatus=0
2016-01-14 15:21:51,452 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784911172_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:51,452 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452784911172_0001_1_01 [B82C2CB46B0F4F988BEDAF348077A2BD] attempt: attempt_1452784911172_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:21:51,452 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B82C2CB46B0F4F988BEDAF348077A2BD
2016-01-14 15:21:51,453 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784911172_0001_1_01 [B82C2CB46B0F4F988BEDAF348077A2BD]
2016-01-14 15:21:51,453 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784911172_0001_1_00 [CCE6ECA6F6484DA1AEA1CC40AF6EC1DF] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:21:51,453 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:51,453 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452784911172_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:21:51,454 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911172_0001_1][Event:TASK_STARTED]: vertexName=B82C2CB46B0F4F988BEDAF348077A2BD, taskId=task_1452784911172_0001_1_01_000000, scheduledTime=1452784911454, launchTime=1452784911454
2016-01-14 15:21:51,454 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784911172_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:51,454 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784911172_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:21:51,454 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784911172_0001_1_01 [B82C2CB46B0F4F988BEDAF348077A2BD]
2016-01-14 15:21:51,454 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3F7415C5BAFF41899AFA67A1E60794F2/B1DA1752CEE8455BAAF7E5B479422B80] count/(1/1) ...latform/testCount2/count2, VertexName: B82C2CB46B0F4F988BEDAF348077A2BD, VertexParallelism: 1, TaskAttemptID:attempt_1452784911172_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CCE6ECA6F6484DA1AEA1CC40AF6EC1DF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=F790FA1E343B402598D4523147C8DDE4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:51,454 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784911172_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:51,455 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:51,455 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:51,455 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=536870912, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452784911431, Process_Duration=2, Process_End_Time=1452784911433, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 15:21:51,455 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784911172_0001_00_000002
2016-01-14 15:21:51,456 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784911172_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:51,457 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784911172_0001_00_000002
2016-01-14 15:21:51,458 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:21:51,458 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:51,458 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:51,459 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784911172_0001_00_000001
2016-01-14 15:21:51,459 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784911172_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:21:51,459 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784911172_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:51,460 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784911172_0001_00_000001, Exiting
2016-01-14 15:21:51,460 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784911172_0001_00_000001
2016-01-14 15:21:51,460 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784911172_0001_00_000001
2016-01-14 15:21:51,462 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911172_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784911172_0001_00_000002, launchTime=1452784911461
2016-01-14 15:21:51,462 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784911172_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:51,462 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:51,463 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784911172_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:51,463 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784911172_0001_00_000001 completed successfully
2016-01-14 15:21:51,463 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784911172_0001_00_000002
2016-01-14 15:21:51,463 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784911172_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:21:51,464 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784911172_0001_1_01_000000_0] to container: [container_1452784911172_0001_00_000002]
2016-01-14 15:21:51,464 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784911172_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:51,464 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784911172_0001_00_000002 given task: attempt_1452784911172_0001_1_01_000000_0
2016-01-14 15:21:51,465 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784911172_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784911172_0001_1_01_000000_0
2016-01-14 15:21:51,465 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784911172_0001_1_01_000000_0] started. Is using containerId: [container_1452784911172_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:21:51,465 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911172_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B82C2CB46B0F4F988BEDAF348077A2BD, taskAttemptId=attempt_1452784911172_0001_1_01_000000_0, startTime=1452784911465, containerId=container_1452784911172_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784911172_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:21:51 Starting to run new task attempt: attempt_1452784911172_0001_1_01_000000_0
2016-01-14 15:21:51,465 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784911172_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:51,465 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:51,466 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784911172_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:51,465 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784911172_0001_1_01_000000_0
2016-01-14 15:21:51,466 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:51,466 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:51,467 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:51,467 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:51,467 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:51,467 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:51,468 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3F7415C5BAFF41899AFA67A1E60794F2/B1DA1752CEE8455BAAF7E5B479422B80] count/(1/1) ...latform/testCount2/count2, VertexName: B82C2CB46B0F4F988BEDAF348077A2BD, VertexParallelism: 1, TaskAttemptID:attempt_1452784911172_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CCE6ECA6F6484DA1AEA1CC40AF6EC1DF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=F790FA1E343B402598D4523147C8DDE4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:51,468 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:51,469 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784911172_0001_1_01_000000_0
2016-01-14 15:21:51,469 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:51,470 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:51,470 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784911172_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:21:51,470 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=CCE6ECA6F6484DA1AEA1CC40AF6EC1DF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:21:51,471 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F790FA1E343B402598D4523147C8DDE4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:21:51,470 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:51,471 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:51,471 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: CCE6ECA6F6484DA1AEA1CC40AF6EC1DF
2016-01-14 15:21:51,473 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:21:51,473 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:21:51,473 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: CCE6ECA6F6484DA1AEA1CC40AF6EC1DF
2016-01-14 15:21:51,473 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: CCE6ECA6F6484DA1AEA1CC40AF6EC1DF
2016-01-14 15:21:51,474 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:51,475 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:51,475 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F790FA1E343B402598D4523147C8DDE4
2016-01-14 15:21:51,476 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:21:51,476 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: F790FA1E343B402598D4523147C8DDE4
2016-01-14 15:21:51,484 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B82C2CB46B0F4F988BEDAF348077A2BD, ordinal: 1
2016-01-14 15:21:51,484 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B82C2CB46B0F4F988BEDAF348077A2BD, mem on start (mb), free: 301, total: 512, max: 672
2016-01-14 15:21:51,485 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:51,485 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:51,485 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:51,487 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:21:51,487 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F790FA1E343B402598D4523147C8DDE4
2016-01-14 15:21:51,487 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:51,488 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:51,488 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:21:51,489 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:21:51,489 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:51,489 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:21:51,489 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:21:51,489 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, CCE6ECA6F6484DA1AEA1CC40AF6EC1DF, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:21:51,489 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F790FA1E343B402598D4523147C8DDE4, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:21:51,489 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:51,490 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: CCE6ECA6F6484DA1AEA1CC40AF6EC1DF
2016-01-14 15:21:51,490 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:21:51,491 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:21:51,491 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: CCE6ECA6F6484DA1AEA1CC40AF6EC1DF with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:21:51,491 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:21:51,491 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:21:51,491 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:21:51,492 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: CCE6ECA6F6484DA1AEA1CC40AF6EC1DF
2016-01-14 15:21:51,495 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: CCE6ECA6F6484DA1AEA1CC40AF6EC1DF being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:51,497 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:21:51,497 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:51,497 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:51,498 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784911172_0001_1_01_000000_0
2016-01-14 15:21:51,498 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452784911172_0001_1_00_000000_0_10068, runDuration: 53542, ]
2016-01-14 15:21:51,499 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452784911172_0001_1_00_000000_0_10068] to DISK_DIRECT, CompressedSize=463, DecompressedSize=459,EndTime=1452784911499, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 15:21:51,499 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : CCE6ECA6F6484DA1AEA1CC40AF6EC1DF
2016-01-14 15:21:51,500 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:21:51,500 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [CCE6ECA6F6484DA1AEA1CC40AF6EC1DF] #11 in 2ms
2016-01-14 15:21:51,500 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [CCE6ECA6F6484DA1AEA1CC40AF6EC1DF] #11, status:false, isInterrupted:false
2016-01-14 15:21:51,501 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:21:51,502 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 463 bytes from disk
2016-01-14 15:21:51,502 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-01-14 15:21:51,502 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:21:51,502 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:21:51,503 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-01-14 15:21:51,503 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : CCE6ECA6F6484DA1AEA1CC40AF6EC1DF
2016-01-14 15:21:51,503 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:21:51,503 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: C272348216784898893E75C4A14A306F
2016-01-14 15:21:51,503 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2"], id: F790FA1E343B402598D4523147C8DDE4
2016-01-14 15:21:51,504 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2"]
2016-01-14 15:21:51,504 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] C272348216784898893E75C4A14A306F, for 1 inputs
2016-01-14 15:21:51,505 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B82C2CB46B0F4F988BEDAF348077A2BD, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:21:51,505 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@1fa38570; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@288d8c4a
2016-01-14 15:21:51,510 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:21:51,511 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:21:51,511 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452784911172_0001_1_01_000000_0
2016-01-14 15:21:51,511 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452784911172_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:21:51,512 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784911172_0001_1_01_000000_0
2016-01-14 15:21:51,513 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B82C2CB46B0F4F988BEDAF348077A2BD, mem on close (mb), free: 299, total: 512, max: 672
2016-01-14 15:21:51,513 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: CCE6ECA6F6484DA1AEA1CC40AF6EC1DF
2016-01-14 15:21:51,513 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:21:51,514 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:21:51,514 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:21:51,514 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:21:51,514 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:21:51 Completed running task attempt: attempt_1452784911172_0001_1_01_000000_0
2016-01-14 15:21:51,516 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784911172_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:51,517 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=536870912, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452784911505, Process_Duration=7, Process_End_Time=1452784911512, Read_Duration=0, Tuples_Read=20, Tuples_Written=17, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=17]]
2016-01-14 15:21:51,517 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:51,517 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911172_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B82C2CB46B0F4F988BEDAF348077A2BD, taskAttemptId=attempt_1452784911172_0001_1_01_000000_0, startTime=1452784911465, finishTime=1452784911517, timeTaken=52, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=536870912, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784911505, Process_Duration=7, Process_End_Time=1452784911512, Read_Duration=0, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-01-14 15:21:51,517 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784911172_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:51,517 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784911172_0001_1_01_000000_0
2016-01-14 15:21:51,518 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:51,518 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:51,518 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:51,518 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:51,519 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784911172_0001_00_000002
2016-01-14 15:21:51,518 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911172_0001_1][Event:TASK_FINISHED]: vertexName=B82C2CB46B0F4F988BEDAF348077A2BD, taskId=task_1452784911172_0001_1_01_000000, startTime=1452784911465, finishTime=1452784911518, timeTaken=53, status=SUCCEEDED, successfulAttemptID=attempt_1452784911172_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=536870912, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784911505, Process_Duration=7, Process_End_Time=1452784911512, Read_Duration=0, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-01-14 15:21:51,520 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784911172_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:51,520 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784911172_0001_1_01 [B82C2CB46B0F4F988BEDAF348077A2BD] : 1
2016-01-14 15:21:51,520 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784911172_0001_1_01 [B82C2CB46B0F4F988BEDAF348077A2BD]
2016-01-14 15:21:51,521 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:51,521 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911172_0001_1][Event:VERTEX_FINISHED]: vertexName=B82C2CB46B0F4F988BEDAF348077A2BD, vertexId=vertex_1452784911172_0001_1_01, initRequestedTime=1452784911348, initedTime=1452784911357, startRequestedTime=1452784911371, startedTime=1452784911371, finishTime=1452784911520, timeTaken=149, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=536870912, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784911505, Process_Duration=7, Process_End_Time=1452784911512, Read_Duration=0, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1452784911465, firstTasksToStart=[ task_1452784911172_0001_1_01_000000 ], lastTaskFinishTime=1452784911517, lastTasksToFinish=[ task_1452784911172_0001_1_01_000000 ], minTaskDuration=52, maxTaskDuration=52, avgTaskDuration=52.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784911172_0001_1_01_000000 ], longestDurationTasks=[ task_1452784911172_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:51,523 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784911172_0001_1_01 [B82C2CB46B0F4F988BEDAF348077A2BD] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:51,523 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784911172_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:51,524 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911172_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784911172_0001_00_000002, stoppedTime=1452784911524, exitStatus=0
2016-01-14 15:21:51,524 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784911172_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:51,524 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784911172_0001_1_01 [B82C2CB46B0F4F988BEDAF348077A2BD] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:21:51,524 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452784911172_0001_1
2016-01-14 15:21:51,524 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:51,525 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911172_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452784911172_0001_1
2016-01-14 15:21:51,525 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: F790FA1E343B402598D4523147C8DDE4 for vertex: vertex_1452784911172_0001_1_01 [B82C2CB46B0F4F988BEDAF348077A2BD]
2016-01-14 15:21:51,530 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784911172_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:21:51,530 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784911172_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:51,530 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784911172_0001_00_000002, Exiting
2016-01-14 15:21:51,530 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784911172_0001_00_000002
2016-01-14 15:21:51,530 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784911172_0001_00_000002
2016-01-14 15:21:51,530 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784911172_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:51,530 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784911172_0001_00_000002 completed successfully
2016-01-14 15:21:51,533 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452784911172_0001_1_00 [CCE6ECA6F6484DA1AEA1CC40AF6EC1DF]
2016-01-14 15:21:51,535 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452784911172_0001_1, queueSize=0
2016-01-14 15:21:51,535 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:21:51,536 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911172_0001_1][Event:DAG_FINISHED]: dagId=dag_1452784911172_0001_1, startTime=1452784911345, finishTime=1452784911533, timeTaken=188, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=940, FILE_BYTES_WRITTEN=1297, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1073741824, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905569822936, Process_Duration=9, Process_End_Time=2905569822945, Read_Duration=1, Tuples_Read=40, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-01-14 15:21:51,537 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452784911172_0001_1 finished with state: SUCCEEDED
2016-01-14 15:21:51,538 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784911172_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:21:51,538 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784911172_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:21:51,538 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452784911172_0001_00_000002
2016-01-14 15:21:51 Completed Dag: dag_1452784911172_0001_1
2016-01-14 15:21:51,541 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452784911172_0001_1, dagState=SUCCEEDED
2016-01-14 15:21:51,542 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784911172_0001_1_post
2016-01-14 15:21:51,543 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:51,543 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:21:51,547 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testCount2/count2, applicationId=application_1452784911172_0001
2016-01-14 15:21:51,551 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:21:51,551 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:21:51,551 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:21:51,551 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:21:51,551 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:21:51,552 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:21:51,552 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:21:51,552 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:21:51,553 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:21:51,553 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:21:51,553 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:21:51,553 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:21:51,553 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:21:51,560 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3F7415C5BAFF41899AFA67A1E60794F2/B1DA1752CEE8455BAAF7E5B479422B80/.tez/application_1452784911172_0001
2016-01-14 15:21:51,562 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] completed in: 00:00.409
2016-01-14 15:21:51,651 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount2/count2
2016-01-14 15:21:51,674 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:21:51,674 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:21:51,713 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-01-14 15:21:51,723 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.024
2016-01-14 15:21:51,723 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:21:51,724 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:21:51,724 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:21:51,740 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-01-14 15:21:51,746 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3
2016-01-14 15:21:51,791 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] starting
2016-01-14 15:21:51,791 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 15:21:51,791 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3"]
2016-01-14 15:21:51,791 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] parallel execution of steps is enabled: false
2016-01-14 15:21:51,792 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executing total steps: 1
2016-01-14 15:21:51,792 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] allocating management threads: 1
2016-01-14 15:21:51,792 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] starting step: (1/1) ...latform/testCount3/count3
2016-01-14 15:21:51,792 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:21:51,804 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tez session mode enabled: true
2016-01-14 15:21:51,809 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:51,809 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:21:51,809 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:21:51,818 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1B0EEF0AAE6049C29BF9B856227176E1/CA6FCAB3B9604BA9895FFFD13246C0A8/.tez/application_1452784911809_0001 doesn't exist and is created
2016-01-14 15:21:51,833 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:21:51,833 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:21:51,834 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1B0EEF0AAE6049C29BF9B856227176E1/CA6FCAB3B9604BA9895FFFD13246C0A8/.tez/application_1452784911809_0001
2016-01-14 15:21:51,837 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452784911809_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:51,846 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:21:51,847 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:21:51,849 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:21:51,849 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:21:51,849 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1B0EEF0AAE6049C29BF9B856227176E1/CA6FCAB3B9604BA9895FFFD13246C0A8/.tez/application_1452784911809_0001/localmode-log-dir
2016-01-14 15:21:51,849 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1B0EEF0AAE6049C29BF9B856227176E1/CA6FCAB3B9604BA9895FFFD13246C0A8/.tez/application_1452784911809_0001/localmode-log-dir/history.txt.appattempt_1452784911809_0001_000000, maxErrors=10
2016-01-14 15:21:51,849 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:21:51,850 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452784911809_0001_000000, appSubmitTime=1452784911836, launchTime=1452784911836
2016-01-14 15:21:51,855 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:21:51,857 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:21:51,858 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:44102
2016-01-14 15:21:51,864 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:21:51,864 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:21:51,865 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:21:51,867 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452784911809_0001_000000, startTime=1452784911867
2016-01-14 15:21:51,867 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:21:51,867 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:21:51,934 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:21:51,934 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:21:51,934 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testCount3/count3, applicationId=application_1452784911809_0001, dagName=[1B0EEF0AAE6049C29BF9B856227176E1/CA6FCAB3B9604BA9895FFFD13246C0A8] count/(1/1) ...latform/testCount3/count3
2016-01-14 15:21:51,950 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [1B0EEF0AAE6049C29BF9B856227176E1/CA6FCAB3B9604BA9895FFFD13246C0A8] count/(1/1) ...latform/testCount3/count3
2016-01-14 15:21:51,952 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452784911809_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1B0EEF0AAE6049C29BF9B856227176E1/CA6FCAB3B9604BA9895FFFD13246C0A8/.tez/application_1452784911809_0001/localmode-log-dir/dag_1452784911809_0001_1.dot
2016-01-14 15:21:51,952 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1B0EEF0AAE6049C29BF9B856227176E1/CA6FCAB3B9604BA9895FFFD13246C0A8/.tez/application_1452784911809_0001/localmode-log-dir/dag_1452784911809_0001_1-tez-dag.pb.txt
2016-01-14 15:21:51,959 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784911809_0001_1
2016-01-14 15:21:51,960 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:51,960 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [1B0EEF0AAE6049C29BF9B856227176E1/CA6FCAB3B9604BA9895FFFD13246C0A8] count/(1/1) ...latform/testCount3/count3
2016-01-14 15:21:51 Running Dag: dag_1452784911809_0001_1
2016-01-14 15:21:51,973 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911809_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452784911809_0001_1, submitTime=1452784911950
2016-01-14 15:21:51,973 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:21:51,974 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FCFF1AE45E1E486F8B94CE7DF28425DE
2016-01-14 15:21:51,974 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:21:51,974 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911809_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452784911809_0001_1, initTime=1452784911973
2016-01-14 15:21:51,974 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:21:51,974 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784911809_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:21:51,975 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911809_0001_1][Event:DAG_STARTED]: dagID=dag_1452784911809_0001_1, startTime=1452784911975
2016-01-14 15:21:51,975 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:21:51,975 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:21:51,975 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784911809_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:21:51,976 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: EAF56391F6AF4CAF805D1FB08B743B65 : {AE2B725EA0E448EEA006998C88237154={InputName=AE2B725EA0E448EEA006998C88237154}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:21:51,976 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: AE2B725EA0E448EEA006998C88237154, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:21:51,976 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452784911809_0001_1_00 [EAF56391F6AF4CAF805D1FB08B743B65]
2016-01-14 15:21:51,976 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452784911809_0001_1_00 [EAF56391F6AF4CAF805D1FB08B743B65]
2016-01-14 15:21:51,976 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452784911809_0001_1_00 [EAF56391F6AF4CAF805D1FB08B743B65]
2016-01-14 15:21:51,976 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452784911809_0001_1_00 [EAF56391F6AF4CAF805D1FB08B743B65]. Starting root input initializers: 1
2016-01-14 15:21:51,977 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784911809_0001_1_00 [EAF56391F6AF4CAF805D1FB08B743B65] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:21:51,977 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: AE2B725EA0E448EEA006998C88237154 on vertex vertex_1452784911809_0001_1_00 [EAF56391F6AF4CAF805D1FB08B743B65]
2016-01-14 15:21:51,977 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testCount3/count3, applicationId=application_1452784911809_0001, dagName=[1B0EEF0AAE6049C29BF9B856227176E1/CA6FCAB3B9604BA9895FFFD13246C0A8] count/(1/1) ...latform/testCount3/count3
2016-01-14 15:21:51,979 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:21:51,979 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input AE2B725EA0E448EEA006998C88237154 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:21:51,979 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:21:51,980 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:21:51,981 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-01-14 15:21:51,982 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: AE2B725EA0E448EEA006998C88237154 on vertex vertex_1452784911809_0001_1_00 [EAF56391F6AF4CAF805D1FB08B743B65]
2016-01-14 15:21:51,977 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452784911809_0001_1_01 [FCFF1AE45E1E486F8B94CE7DF28425DE]
2016-01-14 15:21:51,987 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:21:51,987 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452784911809_0001_1_01 [FCFF1AE45E1E486F8B94CE7DF28425DE]
2016-01-14 15:21:51,987 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452784911809_0001_1_01 [FCFF1AE45E1E486F8B94CE7DF28425DE]
2016-01-14 15:21:51,987 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452784911809_0001_1_01 [FCFF1AE45E1E486F8B94CE7DF28425DE]
2016-01-14 15:21:51,987 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=716F6F90E3C348A9BBB026BD27347463, vertexId=vertex_1452784911809_0001_1_01 [FCFF1AE45E1E486F8B94CE7DF28425DE], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:21:51,987 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=716F6F90E3C348A9BBB026BD27347463, vertexId=vertex_1452784911809_0001_1_01 [FCFF1AE45E1E486F8B94CE7DF28425DE]
2016-01-14 15:21:51,990 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=716F6F90E3C348A9BBB026BD27347463, vertexName=FCFF1AE45E1E486F8B94CE7DF28425DE, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:21:51,990 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=716F6F90E3C348A9BBB026BD27347463, vertexId=vertex_1452784911809_0001_1_01 [FCFF1AE45E1E486F8B94CE7DF28425DE]
2016-01-14 15:21:51,991 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:51,992 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911809_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FCFF1AE45E1E486F8B94CE7DF28425DE, vertexId=vertex_1452784911809_0001_1_01, initRequestedTime=1452784911977, initedTime=1452784911991, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:21:51,992 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784911809_0001_1_01 [FCFF1AE45E1E486F8B94CE7DF28425DE] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:21:51,992 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784911809_0001_1_00 [EAF56391F6AF4CAF805D1FB08B743B65] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:51,993 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {AE2B725EA0E448EEA006998C88237154=forAllWorkUnits=true, update=[1]}
2016-01-14 15:21:51,993 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452784911809_0001_1_00 [EAF56391F6AF4CAF805D1FB08B743B65] parallelism set to 1
2016-01-14 15:21:51,994 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784911809_0001_1_00 [EAF56391F6AF4CAF805D1FB08B743B65] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:51,994 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911809_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784911809_0001_1_00, eventCount=1
2016-01-14 15:21:51,994 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911809_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EAF56391F6AF4CAF805D1FB08B743B65, vertexId=vertex_1452784911809_0001_1_00, initRequestedTime=1452784911976, initedTime=1452784911994, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:21:51,995 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784911809_0001_1_00 [EAF56391F6AF4CAF805D1FB08B743B65] with distanceFromRoot: 0
2016-01-14 15:21:51,995 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] submitted tez dag to app master: application_1452784911809_0001, with dag id: dag_1452784911809_0001_1
2016-01-14 15:21:51,994 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:51,995 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784911809_0001_1_00 [EAF56391F6AF4CAF805D1FB08B743B65] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:21:51,996 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in EAF56391F6AF4CAF805D1FB08B743B65
2016-01-14 15:21:51,996 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784911809_0001_1_00 [EAF56391F6AF4CAF805D1FB08B743B65]
2016-01-14 15:21:51,997 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911809_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784911809_0001_1_00, startRequestedTime=1452784911996, startedTime=1452784911996
2016-01-14 15:21:51,997 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:51,998 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784911809_0001_1_00 [EAF56391F6AF4CAF805D1FB08B743B65] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:52,000 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:52,000 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911809_0001_1][Event:TASK_STARTED]: vertexName=EAF56391F6AF4CAF805D1FB08B743B65, taskId=task_1452784911809_0001_1_00_000000, scheduledTime=1452784912000, launchTime=1452784912000
2016-01-14 15:21:52,001 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784911809_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:52,001 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452784911809_0001_1_00 for vertex: vertex_1452784911809_0001_1_01 [FCFF1AE45E1E486F8B94CE7DF28425DE] numStartedSources: 1 numSources: 1
2016-01-14 15:21:52,001 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784911809_0001_1_01 [FCFF1AE45E1E486F8B94CE7DF28425DE] with distanceFromRoot: 1
2016-01-14 15:21:52,001 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784911809_0001_1_00 [EAF56391F6AF4CAF805D1FB08B743B65]
2016-01-14 15:21:52,002 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: EAF56391F6AF4CAF805D1FB08B743B65 in vertex: FCFF1AE45E1E486F8B94CE7DF28425DE
2016-01-14 15:21:52,002 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: FCFF1AE45E1E486F8B94CE7DF28425DE with 1 source tasks and 1 pending tasks
2016-01-14 15:21:52,003 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:52,003 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911809_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784911809_0001_1_01, startRequestedTime=1452784912002, startedTime=1452784912002
2016-01-14 15:21:52,003 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784911809_0001_1_01 [FCFF1AE45E1E486F8B94CE7DF28425DE] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:52,003 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1B0EEF0AAE6049C29BF9B856227176E1/CA6FCAB3B9604BA9895FFFD13246C0A8] count/(1/1) ...latform/testCount3/count3, VertexName: EAF56391F6AF4CAF805D1FB08B743B65, VertexParallelism: 1, TaskAttemptID:attempt_1452784911809_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AE2B725EA0E448EEA006998C88237154, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FCFF1AE45E1E486F8B94CE7DF28425DE, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:21:52,004 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784911809_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:52,004 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:52,005 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:21:52,005 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:21:52,005 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:21:52,006 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784911809_0001_00_000001
2016-01-14 15:21:52,006 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784911809_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:52,007 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784911809_0001_00_000001
2016-01-14 15:21:52,008 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:21:52,008 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784911809_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:52,008 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911809_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784911809_0001_00_000001, launchTime=1452784912008
2016-01-14 15:21:52,008 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:52,009 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784911809_0001_00_000001
2016-01-14 15:21:52,010 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784911809_0001_1_00_000000_0] to container: [container_1452784911809_0001_00_000001]
2016-01-14 15:21:52,010 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784911809_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:52,010 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784911809_0001_00_000001 given task: attempt_1452784911809_0001_1_00_000000_0
2016-01-14 15:21:52,010 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784911809_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784911809_0001_1_00_000000_0
2016-01-14 15:21:52,010 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784911809_0001_1_00_000000_0] started. Is using containerId: [container_1452784911809_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:21:52,011 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911809_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EAF56391F6AF4CAF805D1FB08B743B65, taskAttemptId=attempt_1452784911809_0001_1_00_000000_0, startTime=1452784912010, containerId=container_1452784911809_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784911809_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:21:52,011 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:52 Starting to run new task attempt: attempt_1452784911809_0001_1_00_000000_0
2016-01-14 15:21:52,011 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784911809_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:52,011 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784911809_0001_1_00_000000_0
2016-01-14 15:21:52,011 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:52,012 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:52,012 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:52,012 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:52,012 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:52,012 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:52,012 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1B0EEF0AAE6049C29BF9B856227176E1/CA6FCAB3B9604BA9895FFFD13246C0A8] count/(1/1) ...latform/testCount3/count3, VertexName: EAF56391F6AF4CAF805D1FB08B743B65, VertexParallelism: 1, TaskAttemptID:attempt_1452784911809_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AE2B725EA0E448EEA006998C88237154, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FCFF1AE45E1E486F8B94CE7DF28425DE, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:21:52,012 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:52,013 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784911809_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:52,014 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784911809_0001_1_00_000000_0
2016-01-14 15:21:52,014 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:52,015 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:52,015 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784911809_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:21:52,016 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=AE2B725EA0E448EEA006998C88237154, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:21:52,017 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:52,017 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: AE2B725EA0E448EEA006998C88237154
2016-01-14 15:21:52,018 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FCFF1AE45E1E486F8B94CE7DF28425DE, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:21:52,018 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:52,019 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:52,019 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FCFF1AE45E1E486F8B94CE7DF28425DE
2016-01-14 15:21:52,018 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: AE2B725EA0E448EEA006998C88237154
2016-01-14 15:21:52,020 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:21:52,020 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: FCFF1AE45E1E486F8B94CE7DF28425DE
2016-01-14 15:21:52,021 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FCFF1AE45E1E486F8B94CE7DF28425DE
2016-01-14 15:21:52,021 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:52,022 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:21:52,023 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: AE2B725EA0E448EEA006998C88237154
2016-01-14 15:21:52,023 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: AE2B725EA0E448EEA006998C88237154
2016-01-14 15:21:52,031 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EAF56391F6AF4CAF805D1FB08B743B65, ordinal: 0
2016-01-14 15:21:52,031 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EAF56391F6AF4CAF805D1FB08B743B65, mem on start (mb), free: 377, total: 592, max: 672
2016-01-14 15:21:52,031 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:52,031 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:52,031 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:52,031 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:52,032 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:52,032 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:21:52,032 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:52,032 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:21:52,032 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:21:52,032 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:21:52,033 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, AE2B725EA0E448EEA006998C88237154, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:21:52,033 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FCFF1AE45E1E486F8B94CE7DF28425DE, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:21:52,033 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:52,033 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: AE2B725EA0E448EEA006998C88237154 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:52,033 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:21:52,033 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:52,033 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: AE2B725EA0E448EEA006998C88237154
2016-01-14 15:21:52,034 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:52,034 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: AE2B725EA0E448EEA006998C88237154
2016-01-14 15:21:52,034 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:52,034 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784911809_0001_1_00_000000_0
2016-01-14 15:21:52,036 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:21:52,036 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@46ece059
2016-01-14 15:21:52,037 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@46ece059
2016-01-14 15:21:52,037 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:21:52,037 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:21:52,037 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 15:21:52,038 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: AE2B725EA0E448EEA006998C88237154
2016-01-14 15:21:52,038 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: 43EEB738D2214F3EA83C779FAB940EB0
2016-01-14 15:21:52,039 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] 43EEB738D2214F3EA83C779FAB940EB0
2016-01-14 15:21:52,039 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@78acc95c; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@17d7f4e9; comparator=cascading.tuple.hadoop.util.TupleComparator@630ff939; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:21:52,040 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:21:52,040 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:21:52,040 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:21:52,054 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:21:52,055 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:21:52,055 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:21:52,055 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:21:52,055 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:21:52,056 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 15:21:52,056 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:52,057 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EAF56391F6AF4CAF805D1FB08B743B65, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:21:52,060 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784911809_0001_1_00_000000_0
2016-01-14 15:21:52,060 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EAF56391F6AF4CAF805D1FB08B743B65, mem on close (mb), free: 275, total: 592, max: 672
2016-01-14 15:21:52,060 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:21:52,060 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:21:52,060 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2016-01-14 15:21:52,060 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 15:21:52,064 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:21:52 Completed running task attempt: attempt_1452784911809_0001_1_00_000000_0
2016-01-14 15:21:52,067 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784911809_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:52,069 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911809_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784911809_0001_1_00, eventCount=1
2016-01-14 15:21:52,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=620756992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452784912057, Process_Duration=3, Process_End_Time=1452784912060, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 15:21:52,070 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:52,070 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911809_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EAF56391F6AF4CAF805D1FB08B743B65, taskAttemptId=attempt_1452784911809_0001_1_00_000000_0, startTime=1452784912010, finishTime=1452784912069, timeTaken=59, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=620756992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784912057, Process_Duration=3, Process_End_Time=1452784912060, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 15:21:52,070 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784911809_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:52,070 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:52,071 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784911809_0001_00_000001
2016-01-14 15:21:52,071 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:52,070 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784911809_0001_1_00_000000_0
2016-01-14 15:21:52,071 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:52,072 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911809_0001_1][Event:TASK_FINISHED]: vertexName=EAF56391F6AF4CAF805D1FB08B743B65, taskId=task_1452784911809_0001_1_00_000000, startTime=1452784912010, finishTime=1452784912072, timeTaken=62, status=SUCCEEDED, successfulAttemptID=attempt_1452784911809_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=620756992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784912057, Process_Duration=3, Process_End_Time=1452784912060, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 15:21:52,072 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784911809_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:52,073 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:52,074 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784911809_0001_1_00 [EAF56391F6AF4CAF805D1FB08B743B65] : 1
2016-01-14 15:21:52,074 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784911809_0001_1_00 [EAF56391F6AF4CAF805D1FB08B743B65]
2016-01-14 15:21:52,075 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911809_0001_1][Event:VERTEX_FINISHED]: vertexName=EAF56391F6AF4CAF805D1FB08B743B65, vertexId=vertex_1452784911809_0001_1_00, initRequestedTime=1452784911976, initedTime=1452784911994, startRequestedTime=1452784911996, startedTime=1452784911996, finishTime=1452784912074, timeTaken=78, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=620756992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784912057, Process_Duration=3, Process_End_Time=1452784912060, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452784912010, firstTasksToStart=[ task_1452784911809_0001_1_00_000000 ], lastTaskFinishTime=1452784912069, lastTasksToFinish=[ task_1452784911809_0001_1_00_000000 ], minTaskDuration=59, maxTaskDuration=59, avgTaskDuration=59.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784911809_0001_1_00_000000 ], longestDurationTasks=[ task_1452784911809_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:52,076 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784911809_0001_1_00 [EAF56391F6AF4CAF805D1FB08B743B65] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:52,076 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784911809_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:52,076 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452784911809_0001_1_01 [FCFF1AE45E1E486F8B94CE7DF28425DE] attempt: attempt_1452784911809_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:21:52,077 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: FCFF1AE45E1E486F8B94CE7DF28425DE
2016-01-14 15:21:52,077 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784911809_0001_1_01 [FCFF1AE45E1E486F8B94CE7DF28425DE]
2016-01-14 15:21:52,077 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911809_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784911809_0001_00_000001, stoppedTime=1452784912077, exitStatus=0
2016-01-14 15:21:52,075 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:52,078 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:52,079 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784911809_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:52,079 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784911809_0001_1_00 [EAF56391F6AF4CAF805D1FB08B743B65] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:21:52,080 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911809_0001_1][Event:TASK_STARTED]: vertexName=FCFF1AE45E1E486F8B94CE7DF28425DE, taskId=task_1452784911809_0001_1_01_000000, scheduledTime=1452784912079, launchTime=1452784912079
2016-01-14 15:21:52,080 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784911809_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:52,080 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784911809_0001_1_01 [FCFF1AE45E1E486F8B94CE7DF28425DE]
2016-01-14 15:21:52,080 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:52,080 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452784911809_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:21:52,081 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784911809_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:21:52,081 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1B0EEF0AAE6049C29BF9B856227176E1/CA6FCAB3B9604BA9895FFFD13246C0A8] count/(1/1) ...latform/testCount3/count3, VertexName: FCFF1AE45E1E486F8B94CE7DF28425DE, VertexParallelism: 1, TaskAttemptID:attempt_1452784911809_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EAF56391F6AF4CAF805D1FB08B743B65, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=716F6F90E3C348A9BBB026BD27347463, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:52,081 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784911809_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:21:52,082 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784911809_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:52,082 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784911809_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:52,082 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784911809_0001_00_000001, Exiting
2016-01-14 15:21:52,082 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784911809_0001_00_000001
2016-01-14 15:21:52,082 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784911809_0001_00_000001
2016-01-14 15:21:52,082 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:52,084 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784911809_0001_00_000002
2016-01-14 15:21:52,084 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784911809_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:52,085 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784911809_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:52,085 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784911809_0001_00_000001 completed successfully
2016-01-14 15:21:52,085 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784911809_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:21:52,086 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784911809_0001_00_000002
2016-01-14 15:21:52,086 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 15:21:52,090 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784911809_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:52,090 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911809_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784911809_0001_00_000002, launchTime=1452784912090
2016-01-14 15:21:52,090 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:52,091 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784911809_0001_00_000002
2016-01-14 15:21:52,092 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784911809_0001_1_01_000000_0] to container: [container_1452784911809_0001_00_000002]
2016-01-14 15:21:52,092 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784911809_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:52,093 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784911809_0001_1_01_000000_0] started. Is using containerId: [container_1452784911809_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:21:52,093 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911809_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FCFF1AE45E1E486F8B94CE7DF28425DE, taskAttemptId=attempt_1452784911809_0001_1_01_000000_0, startTime=1452784912093, containerId=container_1452784911809_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784911809_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:21:52,093 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784911809_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:52,093 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:52,093 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784911809_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:52,094 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784911809_0001_00_000002 given task: attempt_1452784911809_0001_1_01_000000_0
2016-01-14 15:21:52,094 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784911809_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784911809_0001_1_01_000000_0
2016-01-14 15:21:52 Starting to run new task attempt: attempt_1452784911809_0001_1_01_000000_0
2016-01-14 15:21:52,095 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784911809_0001_1_01_000000_0
2016-01-14 15:21:52,095 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:52,095 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:52,095 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:52,095 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:52,095 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:52,096 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:52,096 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1B0EEF0AAE6049C29BF9B856227176E1/CA6FCAB3B9604BA9895FFFD13246C0A8] count/(1/1) ...latform/testCount3/count3, VertexName: FCFF1AE45E1E486F8B94CE7DF28425DE, VertexParallelism: 1, TaskAttemptID:attempt_1452784911809_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EAF56391F6AF4CAF805D1FB08B743B65, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=716F6F90E3C348A9BBB026BD27347463, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:52,096 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:52,097 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784911809_0001_1_01_000000_0
2016-01-14 15:21:52,097 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:52,097 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:52,098 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:52,098 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=EAF56391F6AF4CAF805D1FB08B743B65, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:21:52,098 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:52,098 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: EAF56391F6AF4CAF805D1FB08B743B65
2016-01-14 15:21:52,099 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784911809_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:21:52,101 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:21:52,101 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:21:52,102 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: EAF56391F6AF4CAF805D1FB08B743B65
2016-01-14 15:21:52,102 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: EAF56391F6AF4CAF805D1FB08B743B65
2016-01-14 15:21:52,101 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=716F6F90E3C348A9BBB026BD27347463, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:21:52,101 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:52,103 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:52,103 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 716F6F90E3C348A9BBB026BD27347463
2016-01-14 15:21:52,103 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:21:52,104 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 716F6F90E3C348A9BBB026BD27347463
2016-01-14 15:21:52,109 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FCFF1AE45E1E486F8B94CE7DF28425DE, ordinal: 1
2016-01-14 15:21:52,109 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FCFF1AE45E1E486F8B94CE7DF28425DE, mem on start (mb), free: 270, total: 592, max: 672
2016-01-14 15:21:52,109 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:52,109 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:52,109 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:52,113 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:21:52,113 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 716F6F90E3C348A9BBB026BD27347463
2016-01-14 15:21:52,114 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:52,114 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:52,115 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:21:52,115 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:21:52,115 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:52,115 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:21:52,115 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:21:52,115 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, EAF56391F6AF4CAF805D1FB08B743B65, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:21:52,115 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 716F6F90E3C348A9BBB026BD27347463, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:21:52,115 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:52,116 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: EAF56391F6AF4CAF805D1FB08B743B65 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:52,116 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:21:52,116 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:52,116 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: EAF56391F6AF4CAF805D1FB08B743B65
2016-01-14 15:21:52,117 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:21:52,117 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:21:52,117 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: EAF56391F6AF4CAF805D1FB08B743B65 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:21:52,117 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:21:52,118 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:21:52,118 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:21:52,119 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: EAF56391F6AF4CAF805D1FB08B743B65
2016-01-14 15:21:52,119 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:52,120 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784911809_0001_1_01_000000_0
2016-01-14 15:21:52,121 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452784911809_0001_1_00_000000_0_10074, runDuration: 47069, ]
2016-01-14 15:21:52,123 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452784911809_0001_1_00_000000_0_10074] to DISK_DIRECT, CompressedSize=463, DecompressedSize=459,EndTime=1452784912122, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 15:21:52,123 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : EAF56391F6AF4CAF805D1FB08B743B65
2016-01-14 15:21:52,123 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:21:52,123 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [EAF56391F6AF4CAF805D1FB08B743B65] #12 in 2ms
2016-01-14 15:21:52,134 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [EAF56391F6AF4CAF805D1FB08B743B65] #12, status:false, isInterrupted:false
2016-01-14 15:21:52,134 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-01-14 15:21:52,135 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:21:52,136 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 463 bytes from disk
2016-01-14 15:21:52,136 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:21:52,136 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: 43EEB738D2214F3EA83C779FAB940EB0
2016-01-14 15:21:52,136 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:21:52,137 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-01-14 15:21:52,137 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : EAF56391F6AF4CAF805D1FB08B743B65
2016-01-14 15:21:52,137 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:21:52,137 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3"], id: 716F6F90E3C348A9BBB026BD27347463
2016-01-14 15:21:52,138 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3"]
2016-01-14 15:21:52,138 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] 43EEB738D2214F3EA83C779FAB940EB0, for 1 inputs
2016-01-14 15:21:52,138 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FCFF1AE45E1E486F8B94CE7DF28425DE, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:21:52,139 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@6f882f9f; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@55eea8ed
2016-01-14 15:21:52,143 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:21:52,144 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:21:52,144 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452784911809_0001_1_01_000000_0
2016-01-14 15:21:52,144 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452784911809_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:21:52,145 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784911809_0001_1_01_000000_0
2016-01-14 15:21:52,145 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FCFF1AE45E1E486F8B94CE7DF28425DE, mem on close (mb), free: 349, total: 588, max: 672
2016-01-14 15:21:52,145 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: EAF56391F6AF4CAF805D1FB08B743B65
2016-01-14 15:21:52,145 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:21:52,146 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:21:52,146 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:21:52,146 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:21:52,146 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:21:52 Completed running task attempt: attempt_1452784911809_0001_1_01_000000_0
2016-01-14 15:21:52,147 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784911809_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:52,148 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=616562688, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=20, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452784912139, Process_Duration=6, Process_End_Time=1452784912145, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=17]]
2016-01-14 15:21:52,149 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:52,148 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911809_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FCFF1AE45E1E486F8B94CE7DF28425DE, taskAttemptId=attempt_1452784911809_0001_1_01_000000_0, startTime=1452784912093, finishTime=1452784912148, timeTaken=55, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=616562688, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=20, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784912139, Process_Duration=6, Process_End_Time=1452784912145, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-01-14 15:21:52,149 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784911809_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:52,150 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:52,150 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:52,150 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:52,150 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784911809_0001_00_000002
2016-01-14 15:21:52,150 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784911809_0001_1_01_000000_0
2016-01-14 15:21:52,151 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:52,151 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911809_0001_1][Event:TASK_FINISHED]: vertexName=FCFF1AE45E1E486F8B94CE7DF28425DE, taskId=task_1452784911809_0001_1_01_000000, startTime=1452784912093, finishTime=1452784912150, timeTaken=57, status=SUCCEEDED, successfulAttemptID=attempt_1452784911809_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=616562688, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=20, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784912139, Process_Duration=6, Process_End_Time=1452784912145, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-01-14 15:21:52,151 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784911809_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:52,151 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784911809_0001_1_01 [FCFF1AE45E1E486F8B94CE7DF28425DE] : 1
2016-01-14 15:21:52,152 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784911809_0001_1_01 [FCFF1AE45E1E486F8B94CE7DF28425DE]
2016-01-14 15:21:52,153 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:52,153 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911809_0001_1][Event:VERTEX_FINISHED]: vertexName=FCFF1AE45E1E486F8B94CE7DF28425DE, vertexId=vertex_1452784911809_0001_1_01, initRequestedTime=1452784911977, initedTime=1452784911991, startRequestedTime=1452784912002, startedTime=1452784912002, finishTime=1452784912152, timeTaken=150, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=616562688, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=20, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784912139, Process_Duration=6, Process_End_Time=1452784912145, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1452784912093, firstTasksToStart=[ task_1452784911809_0001_1_01_000000 ], lastTaskFinishTime=1452784912148, lastTasksToFinish=[ task_1452784911809_0001_1_01_000000 ], minTaskDuration=55, maxTaskDuration=55, avgTaskDuration=55.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784911809_0001_1_01_000000 ], longestDurationTasks=[ task_1452784911809_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:52,153 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784911809_0001_1_01 [FCFF1AE45E1E486F8B94CE7DF28425DE] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:52,153 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784911809_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:52,153 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911809_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784911809_0001_00_000002, stoppedTime=1452784912153, exitStatus=0
2016-01-14 15:21:52,153 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:52,154 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784911809_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:52,154 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784911809_0001_1_01 [FCFF1AE45E1E486F8B94CE7DF28425DE] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:21:52,154 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452784911809_0001_1
2016-01-14 15:21:52,154 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911809_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452784911809_0001_1
2016-01-14 15:21:52,154 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 716F6F90E3C348A9BBB026BD27347463 for vertex: vertex_1452784911809_0001_1_01 [FCFF1AE45E1E486F8B94CE7DF28425DE]
2016-01-14 15:21:52,161 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784911809_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:21:52,161 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784911809_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:52,161 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784911809_0001_00_000002, Exiting
2016-01-14 15:21:52,161 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784911809_0001_00_000002
2016-01-14 15:21:52,161 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784911809_0001_00_000002
2016-01-14 15:21:52,164 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784911809_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:52,164 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784911809_0001_00_000002 completed successfully
2016-01-14 15:21:52,164 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452784911809_0001_1_00 [EAF56391F6AF4CAF805D1FB08B743B65]
2016-01-14 15:21:52,167 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452784911809_0001_1, queueSize=0
2016-01-14 15:21:52,167 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:21:52,167 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784911809_0001_1][Event:DAG_FINISHED]: dagId=dag_1452784911809_0001_1, startTime=1452784911975, finishTime=1452784912164, timeTaken=189, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=940, FILE_BYTES_WRITTEN=777, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1237319680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=20, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905569824196, Process_Duration=9, Process_End_Time=2905569824205, Read_Duration=3, Tuples_Read=40, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-01-14 15:21:52,167 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452784911809_0001_1 finished with state: SUCCEEDED
2016-01-14 15:21:52,167 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784911809_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:21:52,168 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784911809_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:21:52,168 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452784911809_0001_00_000002
2016-01-14 15:21:52 Completed Dag: dag_1452784911809_0001_1
2016-01-14 15:21:52,169 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452784911809_0001_1, dagState=SUCCEEDED
2016-01-14 15:21:52,170 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784911809_0001_1_post
2016-01-14 15:21:52,170 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:52,170 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:21:52,174 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testCount3/count3, applicationId=application_1452784911809_0001
2016-01-14 15:21:52,177 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:21:52,177 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:21:52,177 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:21:52,177 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:21:52,177 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:21:52,177 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:21:52,178 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:21:52,178 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:21:52,178 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:21:52,178 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:21:52,179 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:21:52,179 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:21:52,179 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:21:52,184 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1B0EEF0AAE6049C29BF9B856227176E1/CA6FCAB3B9604BA9895FFFD13246C0A8/.tez/application_1452784911809_0001
2016-01-14 15:21:52,184 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] completed in: 00:00.393
2016-01-14 15:21:52,251 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount3/count3
2016-01-14 15:21:52,273 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:21:52,273 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:21:52,309 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-01-14 15:21:52,314 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:21:52,314 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:21:52,325 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.028
2016-01-14 15:21:52,325 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:21:52,325 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:21:52,326 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:21:52,346 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-01-14 15:21:52,354 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4
2016-01-14 15:21:52,395 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] starting
2016-01-14 15:21:52,395 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 15:21:52,396 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4"]
2016-01-14 15:21:52,396 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] parallel execution of steps is enabled: false
2016-01-14 15:21:52,396 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executing total steps: 1
2016-01-14 15:21:52,397 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] allocating management threads: 1
2016-01-14 15:21:52,397 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] starting step: (1/1) ...latform/testCount4/count4
2016-01-14 15:21:52,398 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:21:52,408 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tez session mode enabled: true
2016-01-14 15:21:52,414 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:52,414 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:21:52,414 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:21:52,424 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1E7D0A1757094E78AC36EF915CDB1ABA/DBF9F40897594EFEAD4BEFFC19317F46/.tez/application_1452784912414_0001 doesn't exist and is created
2016-01-14 15:21:52,439 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:21:52,439 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:21:52,440 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1E7D0A1757094E78AC36EF915CDB1ABA/DBF9F40897594EFEAD4BEFFC19317F46/.tez/application_1452784912414_0001
2016-01-14 15:21:52,442 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452784912414_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:52,451 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:21:52,452 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:21:52,453 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:21:52,453 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:21:52,453 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1E7D0A1757094E78AC36EF915CDB1ABA/DBF9F40897594EFEAD4BEFFC19317F46/.tez/application_1452784912414_0001/localmode-log-dir
2016-01-14 15:21:52,454 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1E7D0A1757094E78AC36EF915CDB1ABA/DBF9F40897594EFEAD4BEFFC19317F46/.tez/application_1452784912414_0001/localmode-log-dir/history.txt.appattempt_1452784912414_0001_000000, maxErrors=10
2016-01-14 15:21:52,454 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:21:52,454 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452784912414_0001_000000, appSubmitTime=1452784912442, launchTime=1452784912442
2016-01-14 15:21:52,460 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:21:52,460 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:21:52,461 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:48536
2016-01-14 15:21:52,462 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:21:52,468 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:21:52,469 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:21:52,472 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452784912414_0001_000000, startTime=1452784912472
2016-01-14 15:21:52,472 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:21:52,472 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:21:52,540 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:21:52,540 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:21:52,540 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testCount4/count4, applicationId=application_1452784912414_0001, dagName=[1E7D0A1757094E78AC36EF915CDB1ABA/DBF9F40897594EFEAD4BEFFC19317F46] count/(1/1) ...latform/testCount4/count4
2016-01-14 15:21:52,555 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [1E7D0A1757094E78AC36EF915CDB1ABA/DBF9F40897594EFEAD4BEFFC19317F46] count/(1/1) ...latform/testCount4/count4
2016-01-14 15:21:52,557 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452784912414_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1E7D0A1757094E78AC36EF915CDB1ABA/DBF9F40897594EFEAD4BEFFC19317F46/.tez/application_1452784912414_0001/localmode-log-dir/dag_1452784912414_0001_1.dot
2016-01-14 15:21:52,557 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1E7D0A1757094E78AC36EF915CDB1ABA/DBF9F40897594EFEAD4BEFFC19317F46/.tez/application_1452784912414_0001/localmode-log-dir/dag_1452784912414_0001_1-tez-dag.pb.txt
2016-01-14 15:21:52,564 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784912414_0001_1
2016-01-14 15:21:52,564 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:52,564 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [1E7D0A1757094E78AC36EF915CDB1ABA/DBF9F40897594EFEAD4BEFFC19317F46] count/(1/1) ...latform/testCount4/count4
2016-01-14 15:21:52 Running Dag: dag_1452784912414_0001_1
2016-01-14 15:21:52,577 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912414_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452784912414_0001_1, submitTime=1452784912555
2016-01-14 15:21:52,577 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:21:52,579 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D53B213176414E539A366B4C4B27D96A
2016-01-14 15:21:52,579 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:21:52,580 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912414_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452784912414_0001_1, initTime=1452784912578
2016-01-14 15:21:52,580 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784912414_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:21:52,580 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:21:52,581 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912414_0001_1][Event:DAG_STARTED]: dagID=dag_1452784912414_0001_1, startTime=1452784912581
2016-01-14 15:21:52,581 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:21:52,581 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784912414_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:21:52,581 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CA224BC6B99F49ED9ABFB95F8F3D87C1 : {96B6E8F1B07B41B7B60FE6D9E8282705={InputName=96B6E8F1B07B41B7B60FE6D9E8282705}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:21:52,582 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 96B6E8F1B07B41B7B60FE6D9E8282705, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:21:52,582 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452784912414_0001_1_00 [CA224BC6B99F49ED9ABFB95F8F3D87C1]
2016-01-14 15:21:52,582 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:21:52,582 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452784912414_0001_1_00 [CA224BC6B99F49ED9ABFB95F8F3D87C1]
2016-01-14 15:21:52,582 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452784912414_0001_1_00 [CA224BC6B99F49ED9ABFB95F8F3D87C1]
2016-01-14 15:21:52,583 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452784912414_0001_1_00 [CA224BC6B99F49ED9ABFB95F8F3D87C1]. Starting root input initializers: 1
2016-01-14 15:21:52,584 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testCount4/count4, applicationId=application_1452784912414_0001, dagName=[1E7D0A1757094E78AC36EF915CDB1ABA/DBF9F40897594EFEAD4BEFFC19317F46] count/(1/1) ...latform/testCount4/count4
2016-01-14 15:21:52,584 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784912414_0001_1_00 [CA224BC6B99F49ED9ABFB95F8F3D87C1] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:21:52,584 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 96B6E8F1B07B41B7B60FE6D9E8282705 on vertex vertex_1452784912414_0001_1_00 [CA224BC6B99F49ED9ABFB95F8F3D87C1]
2016-01-14 15:21:52,584 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452784912414_0001_1_01 [D53B213176414E539A366B4C4B27D96A]
2016-01-14 15:21:52,586 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:21:52,587 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 96B6E8F1B07B41B7B60FE6D9E8282705 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:21:52,587 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:21:52,588 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:21:52,589 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-01-14 15:21:52,589 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:21:52,590 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452784912414_0001_1_01 [D53B213176414E539A366B4C4B27D96A]
2016-01-14 15:21:52,590 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 96B6E8F1B07B41B7B60FE6D9E8282705 on vertex vertex_1452784912414_0001_1_00 [CA224BC6B99F49ED9ABFB95F8F3D87C1]
2016-01-14 15:21:52,590 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452784912414_0001_1_01 [D53B213176414E539A366B4C4B27D96A]
2016-01-14 15:21:52,591 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452784912414_0001_1_01 [D53B213176414E539A366B4C4B27D96A]
2016-01-14 15:21:52,591 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=814C150C8D1343659F45FD0A603C209D, vertexId=vertex_1452784912414_0001_1_01 [D53B213176414E539A366B4C4B27D96A], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:21:52,591 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=814C150C8D1343659F45FD0A603C209D, vertexId=vertex_1452784912414_0001_1_01 [D53B213176414E539A366B4C4B27D96A]
2016-01-14 15:21:52,593 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=814C150C8D1343659F45FD0A603C209D, vertexName=D53B213176414E539A366B4C4B27D96A, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:21:52,593 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=814C150C8D1343659F45FD0A603C209D, vertexId=vertex_1452784912414_0001_1_01 [D53B213176414E539A366B4C4B27D96A]
2016-01-14 15:21:52,594 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:52,594 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912414_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D53B213176414E539A366B4C4B27D96A, vertexId=vertex_1452784912414_0001_1_01, initRequestedTime=1452784912584, initedTime=1452784912594, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:21:52,595 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784912414_0001_1_01 [D53B213176414E539A366B4C4B27D96A] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:21:52,595 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784912414_0001_1_00 [CA224BC6B99F49ED9ABFB95F8F3D87C1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:52,595 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {96B6E8F1B07B41B7B60FE6D9E8282705=forAllWorkUnits=true, update=[1]}
2016-01-14 15:21:52,595 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452784912414_0001_1_00 [CA224BC6B99F49ED9ABFB95F8F3D87C1] parallelism set to 1
2016-01-14 15:21:52,595 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784912414_0001_1_00 [CA224BC6B99F49ED9ABFB95F8F3D87C1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:52,596 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912414_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784912414_0001_1_00, eventCount=1
2016-01-14 15:21:52,596 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:52,596 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912414_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CA224BC6B99F49ED9ABFB95F8F3D87C1, vertexId=vertex_1452784912414_0001_1_00, initRequestedTime=1452784912581, initedTime=1452784912596, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:21:52,597 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784912414_0001_1_00 [CA224BC6B99F49ED9ABFB95F8F3D87C1] with distanceFromRoot: 0
2016-01-14 15:21:52,597 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784912414_0001_1_00 [CA224BC6B99F49ED9ABFB95F8F3D87C1] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:21:52,597 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CA224BC6B99F49ED9ABFB95F8F3D87C1
2016-01-14 15:21:52,597 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784912414_0001_1_00 [CA224BC6B99F49ED9ABFB95F8F3D87C1]
2016-01-14 15:21:52,598 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:52,598 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912414_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784912414_0001_1_00, startRequestedTime=1452784912597, startedTime=1452784912597
2016-01-14 15:21:52,598 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784912414_0001_1_00 [CA224BC6B99F49ED9ABFB95F8F3D87C1] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:52,598 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:52,599 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912414_0001_1][Event:TASK_STARTED]: vertexName=CA224BC6B99F49ED9ABFB95F8F3D87C1, taskId=task_1452784912414_0001_1_00_000000, scheduledTime=1452784912598, launchTime=1452784912598
2016-01-14 15:21:52,599 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784912414_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:52,599 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452784912414_0001_1_00 for vertex: vertex_1452784912414_0001_1_01 [D53B213176414E539A366B4C4B27D96A] numStartedSources: 1 numSources: 1
2016-01-14 15:21:52,599 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784912414_0001_1_01 [D53B213176414E539A366B4C4B27D96A] with distanceFromRoot: 1
2016-01-14 15:21:52,599 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784912414_0001_1_00 [CA224BC6B99F49ED9ABFB95F8F3D87C1]
2016-01-14 15:21:52,600 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: CA224BC6B99F49ED9ABFB95F8F3D87C1 in vertex: D53B213176414E539A366B4C4B27D96A
2016-01-14 15:21:52,600 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: D53B213176414E539A366B4C4B27D96A with 1 source tasks and 1 pending tasks
2016-01-14 15:21:52,600 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:52,600 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912414_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784912414_0001_1_01, startRequestedTime=1452784912600, startedTime=1452784912600
2016-01-14 15:21:52,601 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784912414_0001_1_01 [D53B213176414E539A366B4C4B27D96A] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:52,601 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1E7D0A1757094E78AC36EF915CDB1ABA/DBF9F40897594EFEAD4BEFFC19317F46] count/(1/1) ...latform/testCount4/count4, VertexName: CA224BC6B99F49ED9ABFB95F8F3D87C1, VertexParallelism: 1, TaskAttemptID:attempt_1452784912414_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=96B6E8F1B07B41B7B60FE6D9E8282705, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D53B213176414E539A366B4C4B27D96A, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:21:52,601 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784912414_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:52,601 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:52,602 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:21:52,602 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:21:52,603 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:21:52,603 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] submitted tez dag to app master: application_1452784912414_0001, with dag id: dag_1452784912414_0001_1
2016-01-14 15:21:52,604 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784912414_0001_00_000001
2016-01-14 15:21:52,604 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784912414_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:52,607 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784912414_0001_00_000001
2016-01-14 15:21:52,607 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:21:52,608 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:52,608 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912414_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784912414_0001_00_000001, launchTime=1452784912608
2016-01-14 15:21:52,609 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784912414_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:52,609 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784912414_0001_00_000001
2016-01-14 15:21:52,610 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784912414_0001_1_00_000000_0] to container: [container_1452784912414_0001_00_000001]
2016-01-14 15:21:52,610 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784912414_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:52,611 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784912414_0001_1_00_000000_0] started. Is using containerId: [container_1452784912414_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:21:52,611 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:52,611 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784912414_0001_00_000001 given task: attempt_1452784912414_0001_1_00_000000_0
2016-01-14 15:21:52,612 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784912414_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784912414_0001_1_00_000000_0
2016-01-14 15:21:52,612 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912414_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CA224BC6B99F49ED9ABFB95F8F3D87C1, taskAttemptId=attempt_1452784912414_0001_1_00_000000_0, startTime=1452784912611, containerId=container_1452784912414_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784912414_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:21:52 Starting to run new task attempt: attempt_1452784912414_0001_1_00_000000_0
2016-01-14 15:21:52,612 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784912414_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:52,613 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784912414_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:52,612 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784912414_0001_1_00_000000_0
2016-01-14 15:21:52,614 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:52,614 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:52,614 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:52,614 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:52,614 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:52,614 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:52,614 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1E7D0A1757094E78AC36EF915CDB1ABA/DBF9F40897594EFEAD4BEFFC19317F46] count/(1/1) ...latform/testCount4/count4, VertexName: CA224BC6B99F49ED9ABFB95F8F3D87C1, VertexParallelism: 1, TaskAttemptID:attempt_1452784912414_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=96B6E8F1B07B41B7B60FE6D9E8282705, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D53B213176414E539A366B4C4B27D96A, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:21:52,614 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:52,615 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784912414_0001_1_00_000000_0
2016-01-14 15:21:52,615 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:52,615 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:52,616 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:52,616 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=96B6E8F1B07B41B7B60FE6D9E8282705, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:21:52,617 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:52,618 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D53B213176414E539A366B4C4B27D96A, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:21:52,618 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:52,618 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D53B213176414E539A366B4C4B27D96A
2016-01-14 15:21:52,618 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 96B6E8F1B07B41B7B60FE6D9E8282705
2016-01-14 15:21:52,619 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 96B6E8F1B07B41B7B60FE6D9E8282705
2016-01-14 15:21:52,619 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:52,618 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784912414_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:21:52,620 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:21:52,620 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: D53B213176414E539A366B4C4B27D96A
2016-01-14 15:21:52,622 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D53B213176414E539A366B4C4B27D96A
2016-01-14 15:21:52,625 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:21:52,626 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 96B6E8F1B07B41B7B60FE6D9E8282705
2016-01-14 15:21:52,626 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 96B6E8F1B07B41B7B60FE6D9E8282705
2016-01-14 15:21:52,629 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CA224BC6B99F49ED9ABFB95F8F3D87C1, ordinal: 0
2016-01-14 15:21:52,629 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CA224BC6B99F49ED9ABFB95F8F3D87C1, mem on start (mb), free: 264, total: 588, max: 672
2016-01-14 15:21:52,629 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:52,630 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:52,630 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:52,630 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:52,632 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:52,632 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:21:52,632 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:52,633 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:21:52,633 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:21:52,633 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:21:52,633 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 96B6E8F1B07B41B7B60FE6D9E8282705, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:21:52,634 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D53B213176414E539A366B4C4B27D96A, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:21:52,634 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:52,634 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 96B6E8F1B07B41B7B60FE6D9E8282705 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:52,634 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 96B6E8F1B07B41B7B60FE6D9E8282705
2016-01-14 15:21:52,635 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:52,635 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 96B6E8F1B07B41B7B60FE6D9E8282705
2016-01-14 15:21:52,635 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:21:52,636 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:52,636 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:52,636 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784912414_0001_1_00_000000_0
2016-01-14 15:21:52,641 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 15:21:52,641 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: 96B6E8F1B07B41B7B60FE6D9E8282705
2016-01-14 15:21:52,641 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: EF88D84EB69F4605ABD44BC736806FCF
2016-01-14 15:21:52,642 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] EF88D84EB69F4605ABD44BC736806FCF
2016-01-14 15:21:52,641 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:21:52,643 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@786bc2a1; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@6ab715ca; comparator=cascading.tuple.hadoop.util.TupleComparator@5630c51f; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:21:52,644 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1e112d3c
2016-01-14 15:21:52,644 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:21:52,644 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:21:52,645 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:21:52,645 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1e112d3c
2016-01-14 15:21:52,646 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:21:52,646 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:21:52,663 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:21:52,663 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:21:52,663 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:21:52,664 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:21:52,664 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:21:52,664 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 15:21:52,665 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:52,665 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CA224BC6B99F49ED9ABFB95F8F3D87C1, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:21:52,667 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784912414_0001_1_00_000000_0
2016-01-14 15:21:52,668 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CA224BC6B99F49ED9ABFB95F8F3D87C1, mem on close (mb), free: 162, total: 588, max: 672
2016-01-14 15:21:52,668 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:21:52,668 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:21:52,668 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2016-01-14 15:21:52,668 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 15:21:52,671 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:21:52 Completed running task attempt: attempt_1452784912414_0001_1_00_000000_0
2016-01-14 15:21:52,674 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784912414_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:52,675 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912414_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784912414_0001_1_00, eventCount=1
2016-01-14 15:21:52,675 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=616562688, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452784912665, Process_Duration=2, Process_End_Time=1452784912667, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 15:21:52,676 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912414_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CA224BC6B99F49ED9ABFB95F8F3D87C1, taskAttemptId=attempt_1452784912414_0001_1_00_000000_0, startTime=1452784912611, finishTime=1452784912675, timeTaken=64, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=616562688, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784912665, Process_Duration=2, Process_End_Time=1452784912667, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 15:21:52,676 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:52,676 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784912414_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:52,676 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:52,676 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:52,677 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:52,676 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784912414_0001_1_00_000000_0
2016-01-14 15:21:52,678 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:52,678 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912414_0001_1][Event:TASK_FINISHED]: vertexName=CA224BC6B99F49ED9ABFB95F8F3D87C1, taskId=task_1452784912414_0001_1_00_000000, startTime=1452784912611, finishTime=1452784912678, timeTaken=67, status=SUCCEEDED, successfulAttemptID=attempt_1452784912414_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=616562688, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784912665, Process_Duration=2, Process_End_Time=1452784912667, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 15:21:52,678 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784912414_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:52,678 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784912414_0001_1_00 [CA224BC6B99F49ED9ABFB95F8F3D87C1] : 1
2016-01-14 15:21:52,678 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784912414_0001_1_00 [CA224BC6B99F49ED9ABFB95F8F3D87C1]
2016-01-14 15:21:52,679 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:52,679 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912414_0001_1][Event:VERTEX_FINISHED]: vertexName=CA224BC6B99F49ED9ABFB95F8F3D87C1, vertexId=vertex_1452784912414_0001_1_00, initRequestedTime=1452784912581, initedTime=1452784912596, startRequestedTime=1452784912597, startedTime=1452784912597, finishTime=1452784912679, timeTaken=82, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=616562688, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784912665, Process_Duration=2, Process_End_Time=1452784912667, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452784912611, firstTasksToStart=[ task_1452784912414_0001_1_00_000000 ], lastTaskFinishTime=1452784912675, lastTasksToFinish=[ task_1452784912414_0001_1_00_000000 ], minTaskDuration=64, maxTaskDuration=64, avgTaskDuration=64.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784912414_0001_1_00_000000 ], longestDurationTasks=[ task_1452784912414_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:52,680 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784912414_0001_1_00 [CA224BC6B99F49ED9ABFB95F8F3D87C1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:52,680 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784912414_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:52,680 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912414_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784912414_0001_00_000001, stoppedTime=1452784912680, exitStatus=0
2016-01-14 15:21:52,680 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784912414_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:52,693 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452784912414_0001_1_01 [D53B213176414E539A366B4C4B27D96A] attempt: attempt_1452784912414_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:21:52,693 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D53B213176414E539A366B4C4B27D96A
2016-01-14 15:21:52,693 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784912414_0001_1_01 [D53B213176414E539A366B4C4B27D96A]
2016-01-14 15:21:52,693 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784912414_0001_1_00 [CA224BC6B99F49ED9ABFB95F8F3D87C1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:21:52,693 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452784912414_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:21:52,694 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912414_0001_1][Event:TASK_STARTED]: vertexName=D53B213176414E539A366B4C4B27D96A, taskId=task_1452784912414_0001_1_01_000000, scheduledTime=1452784912694, launchTime=1452784912694
2016-01-14 15:21:52,694 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784912414_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:52,694 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784912414_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:21:52,695 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784912414_0001_1_01 [D53B213176414E539A366B4C4B27D96A]
2016-01-14 15:21:52,695 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1E7D0A1757094E78AC36EF915CDB1ABA/DBF9F40897594EFEAD4BEFFC19317F46] count/(1/1) ...latform/testCount4/count4, VertexName: D53B213176414E539A366B4C4B27D96A, VertexParallelism: 1, TaskAttemptID:attempt_1452784912414_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CA224BC6B99F49ED9ABFB95F8F3D87C1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=814C150C8D1343659F45FD0A603C209D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:52,695 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784912414_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:52,695 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:52,696 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784912414_0001_00_000002
2016-01-14 15:21:52,697 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784912414_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:52,692 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:52,698 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:52,679 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784912414_0001_00_000001
2016-01-14 15:21:52,698 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784912414_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:21:52,698 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784912414_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:52,698 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784912414_0001_00_000001, Exiting
2016-01-14 15:21:52,698 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784912414_0001_00_000001
2016-01-14 15:21:52,698 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784912414_0001_00_000001
2016-01-14 15:21:52,699 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784912414_0001_00_000002
2016-01-14 15:21:52,700 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784912414_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:52,700 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 15:21:52,702 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784912414_0001_00_000001 completed successfully
2016-01-14 15:21:52,702 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784912414_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:21:52,706 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784912414_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:52,706 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:52,707 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912414_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784912414_0001_00_000002, launchTime=1452784912706
2016-01-14 15:21:52,708 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784912414_0001_00_000002
2016-01-14 15:21:52,709 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784912414_0001_1_01_000000_0] to container: [container_1452784912414_0001_00_000002]
2016-01-14 15:21:52,709 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784912414_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:52,710 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784912414_0001_1_01_000000_0] started. Is using containerId: [container_1452784912414_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:21:52,710 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784912414_0001_00_000002 given task: attempt_1452784912414_0001_1_01_000000_0
2016-01-14 15:21:52,710 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:52,710 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912414_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D53B213176414E539A366B4C4B27D96A, taskAttemptId=attempt_1452784912414_0001_1_01_000000_0, startTime=1452784912709, containerId=container_1452784912414_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784912414_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:21:52,711 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784912414_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:52,711 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784912414_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:52,711 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784912414_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784912414_0001_1_01_000000_0
2016-01-14 15:21:52 Starting to run new task attempt: attempt_1452784912414_0001_1_01_000000_0
2016-01-14 15:21:52,712 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784912414_0001_1_01_000000_0
2016-01-14 15:21:52,712 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:52,712 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:52,712 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:52,712 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:52,712 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:52,712 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:52,712 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1E7D0A1757094E78AC36EF915CDB1ABA/DBF9F40897594EFEAD4BEFFC19317F46] count/(1/1) ...latform/testCount4/count4, VertexName: D53B213176414E539A366B4C4B27D96A, VertexParallelism: 1, TaskAttemptID:attempt_1452784912414_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CA224BC6B99F49ED9ABFB95F8F3D87C1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=814C150C8D1343659F45FD0A603C209D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:52,713 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:52,715 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784912414_0001_1_01_000000_0
2016-01-14 15:21:52,715 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:52,716 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:52,716 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=CA224BC6B99F49ED9ABFB95F8F3D87C1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:21:52,717 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:52,717 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: CA224BC6B99F49ED9ABFB95F8F3D87C1
2016-01-14 15:21:52,719 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:21:52,719 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:21:52,720 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: CA224BC6B99F49ED9ABFB95F8F3D87C1
2016-01-14 15:21:52,720 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: CA224BC6B99F49ED9ABFB95F8F3D87C1
2016-01-14 15:21:52,716 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784912414_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:21:52,717 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=814C150C8D1343659F45FD0A603C209D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:21:52,717 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:52,724 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:52,725 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:52,727 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 814C150C8D1343659F45FD0A603C209D
2016-01-14 15:21:52,727 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:21:52,728 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 814C150C8D1343659F45FD0A603C209D
2016-01-14 15:21:52,731 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D53B213176414E539A366B4C4B27D96A, ordinal: 1
2016-01-14 15:21:52,731 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D53B213176414E539A366B4C4B27D96A, mem on start (mb), free: 252, total: 593, max: 672
2016-01-14 15:21:52,731 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:52,731 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:52,731 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:52,737 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:21:52,738 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 814C150C8D1343659F45FD0A603C209D
2016-01-14 15:21:52,738 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:52,738 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:52,738 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:21:52,738 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:21:52,738 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:52,738 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:21:52,739 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:21:52,739 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, CA224BC6B99F49ED9ABFB95F8F3D87C1, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:21:52,739 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 814C150C8D1343659F45FD0A603C209D, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:21:52,739 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:52,739 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: CA224BC6B99F49ED9ABFB95F8F3D87C1 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:52,739 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:21:52,739 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:52,739 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: CA224BC6B99F49ED9ABFB95F8F3D87C1
2016-01-14 15:21:52,742 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:21:52,743 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:21:52,743 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: CA224BC6B99F49ED9ABFB95F8F3D87C1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:21:52,744 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:21:52,744 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:21:52,744 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:21:52,745 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: CA224BC6B99F49ED9ABFB95F8F3D87C1
2016-01-14 15:21:52,745 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:52,748 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784912414_0001_1_01_000000_0
2016-01-14 15:21:52,748 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452784912414_0001_1_00_000000_0_10080, runDuration: 54632, ]
2016-01-14 15:21:52,750 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-01-14 15:21:52,751 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: EF88D84EB69F4605ABD44BC736806FCF
2016-01-14 15:21:52,751 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4"], id: 814C150C8D1343659F45FD0A603C209D
2016-01-14 15:21:52,751 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4"]
2016-01-14 15:21:52,751 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] EF88D84EB69F4605ABD44BC736806FCF, for 1 inputs
2016-01-14 15:21:52,752 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452784912414_0001_1_00_000000_0_10080] to DISK_DIRECT, CompressedSize=463, DecompressedSize=459,EndTime=1452784912752, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 15:21:52,753 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : CA224BC6B99F49ED9ABFB95F8F3D87C1
2016-01-14 15:21:52,753 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:21:52,753 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [CA224BC6B99F49ED9ABFB95F8F3D87C1] #13 in 1ms
2016-01-14 15:21:52,753 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [CA224BC6B99F49ED9ABFB95F8F3D87C1] #13, status:false, isInterrupted:false
2016-01-14 15:21:52,754 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:21:52,754 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 463 bytes from disk
2016-01-14 15:21:52,754 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:21:52,755 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:21:52,755 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-01-14 15:21:52,755 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : CA224BC6B99F49ED9ABFB95F8F3D87C1
2016-01-14 15:21:52,755 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:21:52,755 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D53B213176414E539A366B4C4B27D96A, all 1 inputs ready in: 00:00:00.003
2016-01-14 15:21:52,756 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@7ad0fb98; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@2e35f726
2016-01-14 15:21:52,759 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:21:52,760 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:21:52,760 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452784912414_0001_1_01_000000_0
2016-01-14 15:21:52,760 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452784912414_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:21:52,761 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784912414_0001_1_01_000000_0
2016-01-14 15:21:52,761 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D53B213176414E539A366B4C4B27D96A, mem on close (mb), free: 249, total: 593, max: 672
2016-01-14 15:21:52,761 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: CA224BC6B99F49ED9ABFB95F8F3D87C1
2016-01-14 15:21:52,762 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:21:52,762 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:21:52,762 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:21:52,763 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:21:52,763 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:21:52 Completed running task attempt: attempt_1452784912414_0001_1_01_000000_0
2016-01-14 15:21:52,764 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784912414_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:52,765 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=622329856, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452784912756, Process_Duration=5, Process_End_Time=1452784912761, Read_Duration=0, Tuples_Read=20, Tuples_Written=17, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=17]]
2016-01-14 15:21:52,765 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:52,765 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912414_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D53B213176414E539A366B4C4B27D96A, taskAttemptId=attempt_1452784912414_0001_1_01_000000_0, startTime=1452784912709, finishTime=1452784912765, timeTaken=56, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=622329856, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784912756, Process_Duration=5, Process_End_Time=1452784912761, Read_Duration=0, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-01-14 15:21:52,765 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:52,765 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:52,765 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784912414_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:52,766 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784912414_0001_1_01_000000_0
2016-01-14 15:21:52,767 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912414_0001_1][Event:TASK_FINISHED]: vertexName=D53B213176414E539A366B4C4B27D96A, taskId=task_1452784912414_0001_1_01_000000, startTime=1452784912709, finishTime=1452784912766, timeTaken=57, status=SUCCEEDED, successfulAttemptID=attempt_1452784912414_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=622329856, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784912756, Process_Duration=5, Process_End_Time=1452784912761, Read_Duration=0, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-01-14 15:21:52,767 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784912414_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:52,767 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784912414_0001_1_01 [D53B213176414E539A366B4C4B27D96A] : 1
2016-01-14 15:21:52,767 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784912414_0001_1_01 [D53B213176414E539A366B4C4B27D96A]
2016-01-14 15:21:52,769 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912414_0001_1][Event:VERTEX_FINISHED]: vertexName=D53B213176414E539A366B4C4B27D96A, vertexId=vertex_1452784912414_0001_1_01, initRequestedTime=1452784912584, initedTime=1452784912594, startRequestedTime=1452784912600, startedTime=1452784912600, finishTime=1452784912768, timeTaken=168, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=622329856, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784912756, Process_Duration=5, Process_End_Time=1452784912761, Read_Duration=0, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1452784912709, firstTasksToStart=[ task_1452784912414_0001_1_01_000000 ], lastTaskFinishTime=1452784912765, lastTasksToFinish=[ task_1452784912414_0001_1_01_000000 ], minTaskDuration=56, maxTaskDuration=56, avgTaskDuration=56.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784912414_0001_1_01_000000 ], longestDurationTasks=[ task_1452784912414_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:52,769 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784912414_0001_1_01 [D53B213176414E539A366B4C4B27D96A] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:52,769 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784912414_0001_1_01 [D53B213176414E539A366B4C4B27D96A] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:21:52,769 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452784912414_0001_1
2016-01-14 15:21:52,769 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912414_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452784912414_0001_1
2016-01-14 15:21:52,769 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 814C150C8D1343659F45FD0A603C209D for vertex: vertex_1452784912414_0001_1_01 [D53B213176414E539A366B4C4B27D96A]
2016-01-14 15:21:52,770 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:52,770 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:52,771 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:52,772 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784912414_0001_00_000002
2016-01-14 15:21:52,777 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452784912414_0001_1_00 [CA224BC6B99F49ED9ABFB95F8F3D87C1]
2016-01-14 15:21:52,779 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452784912414_0001_1, queueSize=0
2016-01-14 15:21:52,779 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:21:52,780 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912414_0001_1][Event:DAG_FINISHED]: dagId=dag_1452784912414_0001_1, startTime=1452784912581, finishTime=1452784912777, timeTaken=196, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=940, FILE_BYTES_WRITTEN=777, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1238892544, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905569825421, Process_Duration=7, Process_End_Time=2905569825428, Read_Duration=1, Tuples_Read=40, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-01-14 15:21:52,780 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452784912414_0001_1 finished with state: SUCCEEDED
2016-01-14 15:21:52,780 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784912414_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:21:52,780 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784912414_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:52,780 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912414_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784912414_0001_00_000002, stoppedTime=1452784912780, exitStatus=0
2016-01-14 15:21:52,781 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784912414_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:52 Completed Dag: dag_1452784912414_0001_1
2016-01-14 15:21:52,781 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452784912414_0001_1, dagState=SUCCEEDED
2016-01-14 15:21:52,781 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784912414_0001_1_post
2016-01-14 15:21:52,781 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:52,781 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:21:52,783 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:52,783 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784912414_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:21:52,783 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784912414_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:52,785 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784912414_0001_00_000002, Exiting
2016-01-14 15:21:52,785 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784912414_0001_00_000002
2016-01-14 15:21:52,785 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784912414_0001_00_000002
2016-01-14 15:21:52,786 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784912414_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:52,786 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784912414_0001_00_000002 completed successfully
2016-01-14 15:21:52,786 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452784912414_0001_00_000002
2016-01-14 15:21:52,786 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784912414_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:21:52,791 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testCount4/count4, applicationId=application_1452784912414_0001
2016-01-14 15:21:52,792 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:21:52,792 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:21:52,792 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:21:52,793 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:21:52,793 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:21:52,793 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:21:52,794 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:21:52,794 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:21:52,794 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:21:52,794 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:21:52,795 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:21:52,795 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:21:52,795 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:21:52,799 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1E7D0A1757094E78AC36EF915CDB1ABA/DBF9F40897594EFEAD4BEFFC19317F46/.tez/application_1452784912414_0001
2016-01-14 15:21:52,800 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] completed in: 00:00.404
2016-01-14 15:21:52,862 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount4/count4
2016-01-14 15:21:52,884 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:21:52,885 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:21:52,915 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-01-14 15:21:52,925 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-01-14 15:21:52,927 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 15:21:52,927 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 15:21:52,928 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:21:52,940 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-01-14 15:21:52,941 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter
2016-01-14 15:21:52,952 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 15:21:52,952 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 15:21:52,952 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter"]
2016-01-14 15:21:52,953 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 15:21:52,953 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 15:21:52,953 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 15:21:52,955 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...latform/testFilter/filter
2016-01-14 15:21:52,955 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:21:52,966 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 15:21:52,971 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:52,971 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:21:52,971 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:21:52,980 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/328B781A8CA14C019CDF15790A074321/CFA66484E73348DB9539BEE02F5805E6/.tez/application_1452784912971_0001 doesn't exist and is created
2016-01-14 15:21:52,994 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:21:52,994 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:21:52,996 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/328B781A8CA14C019CDF15790A074321/CFA66484E73348DB9539BEE02F5805E6/.tez/application_1452784912971_0001
2016-01-14 15:21:52,999 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452784912971_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:53,010 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:21:53,010 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:21:53,012 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:21:53,013 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:21:53,013 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/328B781A8CA14C019CDF15790A074321/CFA66484E73348DB9539BEE02F5805E6/.tez/application_1452784912971_0001/localmode-log-dir
2016-01-14 15:21:53,013 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/328B781A8CA14C019CDF15790A074321/CFA66484E73348DB9539BEE02F5805E6/.tez/application_1452784912971_0001/localmode-log-dir/history.txt.appattempt_1452784912971_0001_000000, maxErrors=10
2016-01-14 15:21:53,013 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:21:53,015 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452784912971_0001_000000, appSubmitTime=1452784912999, launchTime=1452784912999
2016-01-14 15:21:53,020 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:54904
2016-01-14 15:21:53,020 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:21:53,022 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:21:53,024 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:21:53,024 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:21:53,025 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:21:53,027 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452784912971_0001_000000, startTime=1452784913027
2016-01-14 15:21:53,027 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:21:53,027 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:21:53,095 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:21:53,095 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:21:53,095 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testFilter/filter, applicationId=application_1452784912971_0001, dagName=[328B781A8CA14C019CDF15790A074321/CFA66484E73348DB9539BEE02F5805E6] test/(1/1) ...latform/testFilter/filter
2016-01-14 15:21:53,109 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [328B781A8CA14C019CDF15790A074321/CFA66484E73348DB9539BEE02F5805E6] test/(1/1) ...latform/testFilter/filter
2016-01-14 15:21:53,110 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452784912971_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/328B781A8CA14C019CDF15790A074321/CFA66484E73348DB9539BEE02F5805E6/.tez/application_1452784912971_0001/localmode-log-dir/dag_1452784912971_0001_1.dot
2016-01-14 15:21:53,111 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/328B781A8CA14C019CDF15790A074321/CFA66484E73348DB9539BEE02F5805E6/.tez/application_1452784912971_0001/localmode-log-dir/dag_1452784912971_0001_1-tez-dag.pb.txt
2016-01-14 15:21:53,116 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784912971_0001_1
2016-01-14 15:21:53,116 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:53,116 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [328B781A8CA14C019CDF15790A074321/CFA66484E73348DB9539BEE02F5805E6] test/(1/1) ...latform/testFilter/filter
2016-01-14 15:21:53 Running Dag: dag_1452784912971_0001_1
2016-01-14 15:21:53,126 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912971_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452784912971_0001_1, submitTime=1452784913109
2016-01-14 15:21:53,126 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:21:53,127 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E5CBA663C22040AF86BDFDBBC16843E6
2016-01-14 15:21:53,127 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:21:53,127 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912971_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452784912971_0001_1, initTime=1452784913126
2016-01-14 15:21:53,127 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:21:53,127 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784912971_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:21:53,128 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912971_0001_1][Event:DAG_STARTED]: dagID=dag_1452784912971_0001_1, startTime=1452784913128
2016-01-14 15:21:53,128 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:21:53,128 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784912971_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:21:53,128 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E5CBA663C22040AF86BDFDBBC16843E6 : {9366F81A382A4B5C90C7B426A2EC8CFC={InputName=9366F81A382A4B5C90C7B426A2EC8CFC}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:21:53,128 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 9366F81A382A4B5C90C7B426A2EC8CFC, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:21:53,128 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452784912971_0001_1_00 [E5CBA663C22040AF86BDFDBBC16843E6]
2016-01-14 15:21:53,128 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452784912971_0001_1_00 [E5CBA663C22040AF86BDFDBBC16843E6]
2016-01-14 15:21:53,128 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452784912971_0001_1_00 [E5CBA663C22040AF86BDFDBBC16843E6]
2016-01-14 15:21:53,129 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452784912971_0001_1_00 [E5CBA663C22040AF86BDFDBBC16843E6]. Starting root input initializers: 1
2016-01-14 15:21:53,128 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:21:53,129 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784912971_0001_1_00 [E5CBA663C22040AF86BDFDBBC16843E6] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:21:53,130 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testFilter/filter, applicationId=application_1452784912971_0001, dagName=[328B781A8CA14C019CDF15790A074321/CFA66484E73348DB9539BEE02F5805E6] test/(1/1) ...latform/testFilter/filter
2016-01-14 15:21:53,130 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 9366F81A382A4B5C90C7B426A2EC8CFC on vertex vertex_1452784912971_0001_1_00 [E5CBA663C22040AF86BDFDBBC16843E6]
2016-01-14 15:21:53,132 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:21:53,133 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 9366F81A382A4B5C90C7B426A2EC8CFC asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:21:53,133 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:21:53,134 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:21:53,135 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-01-14 15:21:53,136 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 9366F81A382A4B5C90C7B426A2EC8CFC on vertex vertex_1452784912971_0001_1_00 [E5CBA663C22040AF86BDFDBBC16843E6]
2016-01-14 15:21:53,136 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784912971_0001_1_00 [E5CBA663C22040AF86BDFDBBC16843E6] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:53,136 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {9366F81A382A4B5C90C7B426A2EC8CFC=forAllWorkUnits=true, update=[1]}
2016-01-14 15:21:53,137 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452784912971_0001_1_00 [E5CBA663C22040AF86BDFDBBC16843E6] parallelism set to 1
2016-01-14 15:21:53,137 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784912971_0001_1_00 [E5CBA663C22040AF86BDFDBBC16843E6] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:53,137 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912971_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784912971_0001_1_00, eventCount=1
2016-01-14 15:21:53,137 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452784912971_0001_1_00 [E5CBA663C22040AF86BDFDBBC16843E6]
2016-01-14 15:21:53,137 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=C96CCE9064B7413286D5A216E4D598D9, vertexId=vertex_1452784912971_0001_1_00 [E5CBA663C22040AF86BDFDBBC16843E6], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:21:53,138 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=C96CCE9064B7413286D5A216E4D598D9, vertexId=vertex_1452784912971_0001_1_00 [E5CBA663C22040AF86BDFDBBC16843E6]
2016-01-14 15:21:53,139 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=C96CCE9064B7413286D5A216E4D598D9, vertexName=E5CBA663C22040AF86BDFDBBC16843E6, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:21:53,139 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=C96CCE9064B7413286D5A216E4D598D9, vertexId=vertex_1452784912971_0001_1_00 [E5CBA663C22040AF86BDFDBBC16843E6]
2016-01-14 15:21:53,140 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912971_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E5CBA663C22040AF86BDFDBBC16843E6, vertexId=vertex_1452784912971_0001_1_00, initRequestedTime=1452784913128, initedTime=1452784913139, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:21:53,140 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:53,140 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784912971_0001_1_00 [E5CBA663C22040AF86BDFDBBC16843E6] with distanceFromRoot: 0
2016-01-14 15:21:53,140 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784912971_0001_1_00 [E5CBA663C22040AF86BDFDBBC16843E6] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:21:53,141 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E5CBA663C22040AF86BDFDBBC16843E6
2016-01-14 15:21:53,141 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784912971_0001_1_00 [E5CBA663C22040AF86BDFDBBC16843E6]
2016-01-14 15:21:53,141 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912971_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784912971_0001_1_00, startRequestedTime=1452784913140, startedTime=1452784913140
2016-01-14 15:21:53,141 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:53,141 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784912971_0001_1_00 [E5CBA663C22040AF86BDFDBBC16843E6] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:53,142 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:53,142 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912971_0001_1][Event:TASK_STARTED]: vertexName=E5CBA663C22040AF86BDFDBBC16843E6, taskId=task_1452784912971_0001_1_00_000000, scheduledTime=1452784913142, launchTime=1452784913142
2016-01-14 15:21:53,142 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784912971_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:53,143 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784912971_0001_1_00 [E5CBA663C22040AF86BDFDBBC16843E6]
2016-01-14 15:21:53,143 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [328B781A8CA14C019CDF15790A074321/CFA66484E73348DB9539BEE02F5805E6] test/(1/1) ...latform/testFilter/filter, VertexName: E5CBA663C22040AF86BDFDBBC16843E6, VertexParallelism: 1, TaskAttemptID:attempt_1452784912971_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9366F81A382A4B5C90C7B426A2EC8CFC, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C96CCE9064B7413286D5A216E4D598D9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:53,143 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784912971_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:53,143 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:53,144 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:21:53,145 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:21:53,145 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:21:53,145 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784912971_0001_00_000001
2016-01-14 15:21:53,145 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784912971_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:53,147 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784912971_0001_00_000001
2016-01-14 15:21:53,147 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:21:53,148 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784912971_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:53,148 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912971_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784912971_0001_00_000001, launchTime=1452784913148
2016-01-14 15:21:53,148 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:53,149 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784912971_0001_00_000001
2016-01-14 15:21:53,149 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784912971_0001_1_00_000000_0] to container: [container_1452784912971_0001_00_000001]
2016-01-14 15:21:53,150 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784912971_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:53,150 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784912971_0001_1_00_000000_0] started. Is using containerId: [container_1452784912971_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:21:53,151 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:53,151 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912971_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E5CBA663C22040AF86BDFDBBC16843E6, taskAttemptId=attempt_1452784912971_0001_1_00_000000_0, startTime=1452784913150, containerId=container_1452784912971_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784912971_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:21:53,151 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784912971_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:53,151 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784912971_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:53,152 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784912971_0001_00_000001 given task: attempt_1452784912971_0001_1_00_000000_0
2016-01-14 15:21:53,152 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784912971_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784912971_0001_1_00_000000_0
2016-01-14 15:21:53 Starting to run new task attempt: attempt_1452784912971_0001_1_00_000000_0
2016-01-14 15:21:53,152 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784912971_0001_1_00_000000_0
2016-01-14 15:21:53,153 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:53,153 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:53,153 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:53,153 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:53,153 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452784912971_0001, with dag id: dag_1452784912971_0001_1
2016-01-14 15:21:53,153 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:53,156 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:53,156 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [328B781A8CA14C019CDF15790A074321/CFA66484E73348DB9539BEE02F5805E6] test/(1/1) ...latform/testFilter/filter, VertexName: E5CBA663C22040AF86BDFDBBC16843E6, VertexParallelism: 1, TaskAttemptID:attempt_1452784912971_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9366F81A382A4B5C90C7B426A2EC8CFC, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C96CCE9064B7413286D5A216E4D598D9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:53,156 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:53,156 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784912971_0001_1_00_000000_0
2016-01-14 15:21:53,157 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:53,157 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:53,157 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784912971_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:21:53,158 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:53,158 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=9366F81A382A4B5C90C7B426A2EC8CFC, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:21:53,158 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:53,159 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 9366F81A382A4B5C90C7B426A2EC8CFC
2016-01-14 15:21:53,159 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 9366F81A382A4B5C90C7B426A2EC8CFC
2016-01-14 15:21:53,160 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:53,168 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E5CBA663C22040AF86BDFDBBC16843E6, ordinal: 0
2016-01-14 15:21:53,169 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E5CBA663C22040AF86BDFDBBC16843E6, mem on start (mb), free: 182, total: 593, max: 672
2016-01-14 15:21:53,169 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:53,170 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:53,168 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C96CCE9064B7413286D5A216E4D598D9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:21:53,171 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:53,172 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C96CCE9064B7413286D5A216E4D598D9
2016-01-14 15:21:53,172 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:21:53,169 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:21:53,173 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 9366F81A382A4B5C90C7B426A2EC8CFC
2016-01-14 15:21:53,173 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 9366F81A382A4B5C90C7B426A2EC8CFC
2016-01-14 15:21:53,173 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:53,173 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: C96CCE9064B7413286D5A216E4D598D9
2016-01-14 15:21:53,183 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:21:53,183 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C96CCE9064B7413286D5A216E4D598D9
2016-01-14 15:21:53,183 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:53,183 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:53,183 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:21:53,184 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:53,184 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:21:53,184 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:53,184 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:21:53,184 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-01-14 15:21:53,184 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 9366F81A382A4B5C90C7B426A2EC8CFC, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:21:53,184 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C96CCE9064B7413286D5A216E4D598D9, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:21:53,184 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:53,184 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 9366F81A382A4B5C90C7B426A2EC8CFC being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:53,184 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:21:53,184 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:53,185 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 9366F81A382A4B5C90C7B426A2EC8CFC
2016-01-14 15:21:53,185 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:53,185 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 9366F81A382A4B5C90C7B426A2EC8CFC
2016-01-14 15:21:53,185 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:53,186 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784912971_0001_1_00_000000_0
2016-01-14 15:21:53,188 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:21:53,188 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@22d418e2
2016-01-14 15:21:53,189 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 15:21:53,189 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 9366F81A382A4B5C90C7B426A2EC8CFC
2016-01-14 15:21:53,189 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter"], id: C96CCE9064B7413286D5A216E4D598D9
2016-01-14 15:21:53,190 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter"]
2016-01-14 15:21:53,190 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 15:21:53,190 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:53,190 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@22d418e2
2016-01-14 15:21:53,191 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:21:53,191 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:21:53,192 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E5CBA663C22040AF86BDFDBBC16843E6, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:21:53,194 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:21:53,194 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:21:53,194 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452784912971_0001_1_00_000000_0
2016-01-14 15:21:53,194 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452784912971_0001_1_00_000000_0 given a go for committing the task output.
2016-01-14 15:21:53,195 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784912971_0001_1_00_000000_0
2016-01-14 15:21:53,195 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E5CBA663C22040AF86BDFDBBC16843E6, mem on close (mb), free: 179, total: 593, max: 672
2016-01-14 15:21:53 Completed running task attempt: attempt_1452784912971_0001_1_00_000000_0
2016-01-14 15:21:53,196 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784912971_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:53,197 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:53,197 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912971_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E5CBA663C22040AF86BDFDBBC16843E6, taskAttemptId=attempt_1452784912971_0001_1_00_000000_0, startTime=1452784913150, finishTime=1452784913197, timeTaken=47, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=507, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=622329856, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1452784913192, Process_Duration=3, Process_End_Time=1452784913195, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3
2016-01-14 15:21:53,198 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784912971_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:53,198 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:53,198 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784912971_0001_1_00_000000_0
2016-01-14 15:21:53,199 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:53,199 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912971_0001_1][Event:TASK_FINISHED]: vertexName=E5CBA663C22040AF86BDFDBBC16843E6, taskId=task_1452784912971_0001_1_00_000000, startTime=1452784913150, finishTime=1452784913198, timeTaken=48, status=SUCCEEDED, successfulAttemptID=attempt_1452784912971_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=507, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=622329856, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1452784913192, Process_Duration=3, Process_End_Time=1452784913195, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3
2016-01-14 15:21:53,199 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784912971_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:53,199 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784912971_0001_1_00 [E5CBA663C22040AF86BDFDBBC16843E6] : 1
2016-01-14 15:21:53,199 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784912971_0001_1_00 [E5CBA663C22040AF86BDFDBBC16843E6]
2016-01-14 15:21:53,200 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:53,200 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912971_0001_1][Event:VERTEX_FINISHED]: vertexName=E5CBA663C22040AF86BDFDBBC16843E6, vertexId=vertex_1452784912971_0001_1_00, initRequestedTime=1452784913128, initedTime=1452784913139, startRequestedTime=1452784913140, startedTime=1452784913140, finishTime=1452784913199, timeTaken=59, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=507, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=622329856, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1452784913192, Process_Duration=3, Process_End_Time=1452784913195, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3, vertexStats=firstTaskStartTime=1452784913150, firstTasksToStart=[ task_1452784912971_0001_1_00_000000 ], lastTaskFinishTime=1452784913197, lastTasksToFinish=[ task_1452784912971_0001_1_00_000000 ], minTaskDuration=47, maxTaskDuration=47, avgTaskDuration=47.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784912971_0001_1_00_000000 ], longestDurationTasks=[ task_1452784912971_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:53,200 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784912971_0001_1_00 [E5CBA663C22040AF86BDFDBBC16843E6] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:53,201 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784912971_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:53,201 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=507, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=622329856, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3][cascading.flow.SliceCounters Process_Begin_Time=1452784913192, Process_Duration=3, Process_End_Time=1452784913195, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10, Tuples_Written=3]]
2016-01-14 15:21:53,201 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912971_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784912971_0001_00_000001, stoppedTime=1452784913201, exitStatus=0
2016-01-14 15:21:53,201 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:53,202 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:53,201 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:53,201 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784912971_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:53,202 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784912971_0001_00_000001
2016-01-14 15:21:53,202 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784912971_0001_1_00 [E5CBA663C22040AF86BDFDBBC16843E6] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-01-14 15:21:53,202 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452784912971_0001_1
2016-01-14 15:21:53,202 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784912971_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:21:53,202 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784912971_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:53,202 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784912971_0001_00_000001, Exiting
2016-01-14 15:21:53,203 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784912971_0001_00_000001
2016-01-14 15:21:53,203 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784912971_0001_00_000001
2016-01-14 15:21:53,203 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912971_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452784912971_0001_1
2016-01-14 15:21:53,203 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: C96CCE9064B7413286D5A216E4D598D9 for vertex: vertex_1452784912971_0001_1_00 [E5CBA663C22040AF86BDFDBBC16843E6]
2016-01-14 15:21:53,204 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784912971_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:53,204 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784912971_0001_00_000001 completed successfully
2016-01-14 15:21:53,212 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452784912971_0001_1, queueSize=0
2016-01-14 15:21:53,212 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:21:53,213 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784912971_0001_1][Event:DAG_FINISHED]: dagId=dag_1452784912971_0001_1, startTime=1452784913128, finishTime=1452784913211, timeTaken=83, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=507, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=622329856, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1452784913192, Process_Duration=3, Process_End_Time=1452784913195, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3
2016-01-14 15:21:53,213 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452784912971_0001_1 finished with state: SUCCEEDED
2016-01-14 15:21:53,213 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784912971_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:21:53,213 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452784912971_0001_00_000001
2016-01-14 15:21:53,214 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784912971_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:21:53 Completed Dag: dag_1452784912971_0001_1
2016-01-14 15:21:53,216 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452784912971_0001_1, dagState=SUCCEEDED
2016-01-14 15:21:53,217 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784912971_0001_1_post
2016-01-14 15:21:53,218 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:53,218 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:21:53,219 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testFilter/filter, applicationId=application_1452784912971_0001
2016-01-14 15:21:53,221 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:21:53,221 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:21:53,222 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:21:53,222 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:21:53,222 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:21:53,222 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:21:53,222 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:21:53,223 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:21:53,223 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:21:53,223 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:21:53,223 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:21:53,224 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:21:53,224 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:21:53,228 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/328B781A8CA14C019CDF15790A074321/CFA66484E73348DB9539BEE02F5805E6/.tez/application_1452784912971_0001
2016-01-14 15:21:53,228 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:00.276
2016-01-14 15:21:53,259 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testFilter/filter
2016-01-14 15:21:53,283 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:21:53,283 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:21:53,330 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:21:53,330 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:21:53,333 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.027
2016-01-14 15:21:53,343 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.037
2016-01-14 15:21:53,344 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:21:53,344 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:21:53,344 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:21:53,361 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-01-14 15:21:53,368 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple
2016-01-14 15:21:53,410 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 15:21:53,410 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 15:21:53,411 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple"]
2016-01-14 15:21:53,411 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 15:21:53,411 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 15:21:53,411 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 15:21:53,412 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...latform/testSimple/simple
2016-01-14 15:21:53,412 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:21:53,423 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 15:21:53,427 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:53,428 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:21:53,428 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:21:53,437 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/88A484A83144445F8032F24829241921/BFE2FCCF869A4FABB227C7E7D9814CFB/.tez/application_1452784913428_0001 doesn't exist and is created
2016-01-14 15:21:53,452 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:21:53,453 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:21:53,453 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/88A484A83144445F8032F24829241921/BFE2FCCF869A4FABB227C7E7D9814CFB/.tez/application_1452784913428_0001
2016-01-14 15:21:53,456 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452784913428_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:21:53,465 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:21:53,466 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:21:53,467 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:21:53,467 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:21:53,468 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/88A484A83144445F8032F24829241921/BFE2FCCF869A4FABB227C7E7D9814CFB/.tez/application_1452784913428_0001/localmode-log-dir
2016-01-14 15:21:53,468 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/88A484A83144445F8032F24829241921/BFE2FCCF869A4FABB227C7E7D9814CFB/.tez/application_1452784913428_0001/localmode-log-dir/history.txt.appattempt_1452784913428_0001_000000, maxErrors=10
2016-01-14 15:21:53,468 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:21:53,469 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452784913428_0001_000000, appSubmitTime=1452784913455, launchTime=1452784913455
2016-01-14 15:21:53,473 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:21:53,477 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:21:53,478 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:21:53,478 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:21:53,482 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:36066
2016-01-14 15:21:53,485 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:21:53,488 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452784913428_0001_000000, startTime=1452784913488
2016-01-14 15:21:53,488 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:21:53,488 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:21:53,553 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:21:53,553 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:21:53,553 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testSimple/simple, applicationId=application_1452784913428_0001, dagName=[88A484A83144445F8032F24829241921/BFE2FCCF869A4FABB227C7E7D9814CFB] test/(1/1) ...latform/testSimple/simple
2016-01-14 15:21:53,568 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [88A484A83144445F8032F24829241921/BFE2FCCF869A4FABB227C7E7D9814CFB] test/(1/1) ...latform/testSimple/simple
2016-01-14 15:21:53,570 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452784913428_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/88A484A83144445F8032F24829241921/BFE2FCCF869A4FABB227C7E7D9814CFB/.tez/application_1452784913428_0001/localmode-log-dir/dag_1452784913428_0001_1.dot
2016-01-14 15:21:53,570 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/88A484A83144445F8032F24829241921/BFE2FCCF869A4FABB227C7E7D9814CFB/.tez/application_1452784913428_0001/localmode-log-dir/dag_1452784913428_0001_1-tez-dag.pb.txt
2016-01-14 15:21:53,581 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784913428_0001_1
2016-01-14 15:21:53,581 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:53,581 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [88A484A83144445F8032F24829241921/BFE2FCCF869A4FABB227C7E7D9814CFB] test/(1/1) ...latform/testSimple/simple
2016-01-14 15:21:53 Running Dag: dag_1452784913428_0001_1
2016-01-14 15:21:53,593 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784913428_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452784913428_0001_1, submitTime=1452784913568
2016-01-14 15:21:53,593 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:21:53,594 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D00171769664416C8ECB009AB2D6F4AD
2016-01-14 15:21:53,594 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:21:53,595 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784913428_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452784913428_0001_1, initTime=1452784913593
2016-01-14 15:21:53,595 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:21:53,596 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784913428_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:21:53,597 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784913428_0001_1][Event:DAG_STARTED]: dagID=dag_1452784913428_0001_1, startTime=1452784913597
2016-01-14 15:21:53,597 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:21:53,597 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:21:53,597 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784913428_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:21:53,597 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D78EDD263AA34AD3A59647AA872D1C15 : {2CA376B3B5844B39BA6D0AC7E3B3CABA={InputName=2CA376B3B5844B39BA6D0AC7E3B3CABA}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:21:53,597 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 2CA376B3B5844B39BA6D0AC7E3B3CABA, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:21:53,597 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452784913428_0001_1_00 [D78EDD263AA34AD3A59647AA872D1C15]
2016-01-14 15:21:53,597 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452784913428_0001_1_00 [D78EDD263AA34AD3A59647AA872D1C15]
2016-01-14 15:21:53,597 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452784913428_0001_1_00 [D78EDD263AA34AD3A59647AA872D1C15]
2016-01-14 15:21:53,598 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452784913428_0001_1_00 [D78EDD263AA34AD3A59647AA872D1C15]. Starting root input initializers: 1
2016-01-14 15:21:53,599 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784913428_0001_1_00 [D78EDD263AA34AD3A59647AA872D1C15] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:21:53,599 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 2CA376B3B5844B39BA6D0AC7E3B3CABA on vertex vertex_1452784913428_0001_1_00 [D78EDD263AA34AD3A59647AA872D1C15]
2016-01-14 15:21:53,599 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452784913428_0001_1_01 [D00171769664416C8ECB009AB2D6F4AD]
2016-01-14 15:21:53,600 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testSimple/simple, applicationId=application_1452784913428_0001, dagName=[88A484A83144445F8032F24829241921/BFE2FCCF869A4FABB227C7E7D9814CFB] test/(1/1) ...latform/testSimple/simple
2016-01-14 15:21:53,601 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:21:53,601 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 2CA376B3B5844B39BA6D0AC7E3B3CABA asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:21:53,601 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:21:53,601 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:21:53,602 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:21:53,602 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452784913428_0001_1_01 [D00171769664416C8ECB009AB2D6F4AD]
2016-01-14 15:21:53,602 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452784913428_0001_1_01 [D00171769664416C8ECB009AB2D6F4AD]
2016-01-14 15:21:53,602 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452784913428_0001_1_01 [D00171769664416C8ECB009AB2D6F4AD]
2016-01-14 15:21:53,603 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=1E54D205A69E413FA590B3BB7B4A5D86, vertexId=vertex_1452784913428_0001_1_01 [D00171769664416C8ECB009AB2D6F4AD], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:21:53,603 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-01-14 15:21:53,603 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 2CA376B3B5844B39BA6D0AC7E3B3CABA on vertex vertex_1452784913428_0001_1_00 [D78EDD263AA34AD3A59647AA872D1C15]
2016-01-14 15:21:53,603 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=1E54D205A69E413FA590B3BB7B4A5D86, vertexId=vertex_1452784913428_0001_1_01 [D00171769664416C8ECB009AB2D6F4AD]
2016-01-14 15:21:53,605 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=1E54D205A69E413FA590B3BB7B4A5D86, vertexName=D00171769664416C8ECB009AB2D6F4AD, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:21:53,605 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=1E54D205A69E413FA590B3BB7B4A5D86, vertexId=vertex_1452784913428_0001_1_01 [D00171769664416C8ECB009AB2D6F4AD]
2016-01-14 15:21:53,607 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:53,607 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784913428_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D00171769664416C8ECB009AB2D6F4AD, vertexId=vertex_1452784913428_0001_1_01, initRequestedTime=1452784913599, initedTime=1452784913607, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:21:53,607 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784913428_0001_1_01 [D00171769664416C8ECB009AB2D6F4AD] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:21:53,607 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784913428_0001_1_00 [D78EDD263AA34AD3A59647AA872D1C15] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:53,608 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {2CA376B3B5844B39BA6D0AC7E3B3CABA=forAllWorkUnits=true, update=[1]}
2016-01-14 15:21:53,608 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452784913428_0001_1_00 [D78EDD263AA34AD3A59647AA872D1C15] parallelism set to 1
2016-01-14 15:21:53,608 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452784913428_0001_1_00 [D78EDD263AA34AD3A59647AA872D1C15] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:21:53,609 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784913428_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784913428_0001_1_00, eventCount=1
2016-01-14 15:21:53,609 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:21:53,609 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784913428_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D78EDD263AA34AD3A59647AA872D1C15, vertexId=vertex_1452784913428_0001_1_00, initRequestedTime=1452784913597, initedTime=1452784913609, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:21:53,610 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784913428_0001_1_00 [D78EDD263AA34AD3A59647AA872D1C15] with distanceFromRoot: 0
2016-01-14 15:21:53,610 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784913428_0001_1_00 [D78EDD263AA34AD3A59647AA872D1C15] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:21:53,610 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D78EDD263AA34AD3A59647AA872D1C15
2016-01-14 15:21:53,610 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784913428_0001_1_00 [D78EDD263AA34AD3A59647AA872D1C15]
2016-01-14 15:21:53,611 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:53,611 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784913428_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784913428_0001_1_00, startRequestedTime=1452784913610, startedTime=1452784913610
2016-01-14 15:21:53,611 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784913428_0001_1_00 [D78EDD263AA34AD3A59647AA872D1C15] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:53,612 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:53,612 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784913428_0001_1][Event:TASK_STARTED]: vertexName=D78EDD263AA34AD3A59647AA872D1C15, taskId=task_1452784913428_0001_1_00_000000, scheduledTime=1452784913612, launchTime=1452784913612
2016-01-14 15:21:53,612 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784913428_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:53,613 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452784913428_0001_1_00 for vertex: vertex_1452784913428_0001_1_01 [D00171769664416C8ECB009AB2D6F4AD] numStartedSources: 1 numSources: 1
2016-01-14 15:21:53,613 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452784913428_0001_1_01 [D00171769664416C8ECB009AB2D6F4AD] with distanceFromRoot: 1
2016-01-14 15:21:53,613 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784913428_0001_1_00 [D78EDD263AA34AD3A59647AA872D1C15]
2016-01-14 15:21:53,614 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D78EDD263AA34AD3A59647AA872D1C15 in vertex: D00171769664416C8ECB009AB2D6F4AD
2016-01-14 15:21:53,614 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: D00171769664416C8ECB009AB2D6F4AD with 1 source tasks and 1 pending tasks
2016-01-14 15:21:53,614 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:21:53,614 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784913428_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452784913428_0001_1_01, startRequestedTime=1452784913613, startedTime=1452784913613
2016-01-14 15:21:53,615 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784913428_0001_1_01 [D00171769664416C8ECB009AB2D6F4AD] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:21:53,615 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [88A484A83144445F8032F24829241921/BFE2FCCF869A4FABB227C7E7D9814CFB] test/(1/1) ...latform/testSimple/simple, VertexName: D78EDD263AA34AD3A59647AA872D1C15, VertexParallelism: 1, TaskAttemptID:attempt_1452784913428_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2CA376B3B5844B39BA6D0AC7E3B3CABA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D00171769664416C8ECB009AB2D6F4AD, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:21:53,615 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784913428_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:53,616 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:53,616 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:21:53,617 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:21:53,617 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:21:53,617 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784913428_0001_00_000001
2016-01-14 15:21:53,617 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784913428_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:53,618 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452784913428_0001, with dag id: dag_1452784913428_0001_1
2016-01-14 15:21:53,619 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784913428_0001_00_000001
2016-01-14 15:21:53,619 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:21:53,620 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784913428_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:53,620 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:53,620 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784913428_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784913428_0001_00_000001, launchTime=1452784913620
2016-01-14 15:21:53,621 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784913428_0001_00_000001
2016-01-14 15:21:53,621 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784913428_0001_1_00_000000_0] to container: [container_1452784913428_0001_00_000001]
2016-01-14 15:21:53,621 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784913428_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:53,622 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784913428_0001_1_00_000000_0] started. Is using containerId: [container_1452784913428_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:21:53,622 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784913428_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D78EDD263AA34AD3A59647AA872D1C15, taskAttemptId=attempt_1452784913428_0001_1_00_000000_0, startTime=1452784913622, containerId=container_1452784913428_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784913428_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:21:53,622 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784913428_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:53,622 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:53,622 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784913428_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:53,623 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784913428_0001_00_000001 given task: attempt_1452784913428_0001_1_00_000000_0
2016-01-14 15:21:53,623 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784913428_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784913428_0001_1_00_000000_0
2016-01-14 15:21:53 Starting to run new task attempt: attempt_1452784913428_0001_1_00_000000_0
2016-01-14 15:21:53,624 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784913428_0001_1_00_000000_0
2016-01-14 15:21:53,624 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:53,624 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:53,624 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:53,624 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:53,624 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:53,624 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:53,624 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [88A484A83144445F8032F24829241921/BFE2FCCF869A4FABB227C7E7D9814CFB] test/(1/1) ...latform/testSimple/simple, VertexName: D78EDD263AA34AD3A59647AA872D1C15, VertexParallelism: 1, TaskAttemptID:attempt_1452784913428_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2CA376B3B5844B39BA6D0AC7E3B3CABA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D00171769664416C8ECB009AB2D6F4AD, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:21:53,625 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:53,625 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784913428_0001_1_00_000000_0
2016-01-14 15:21:53,625 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:53,626 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:53,635 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:53,636 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784913428_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:21:53,636 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D00171769664416C8ECB009AB2D6F4AD, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:21:53,638 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:53,638 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:53,639 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D00171769664416C8ECB009AB2D6F4AD
2016-01-14 15:21:53,641 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:21:53,641 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: D00171769664416C8ECB009AB2D6F4AD
2016-01-14 15:21:53,641 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D00171769664416C8ECB009AB2D6F4AD
2016-01-14 15:21:53,643 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=2CA376B3B5844B39BA6D0AC7E3B3CABA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:21:53,643 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:53,643 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 2CA376B3B5844B39BA6D0AC7E3B3CABA
2016-01-14 15:21:53,643 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 2CA376B3B5844B39BA6D0AC7E3B3CABA
2016-01-14 15:21:53,645 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D78EDD263AA34AD3A59647AA872D1C15, ordinal: 0
2016-01-14 15:21:53,645 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D78EDD263AA34AD3A59647AA872D1C15, mem on start (mb), free: 198, total: 590, max: 672
2016-01-14 15:21:53,645 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:53,645 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:53,645 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:53,646 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:21:53,646 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 2CA376B3B5844B39BA6D0AC7E3B3CABA
2016-01-14 15:21:53,646 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 2CA376B3B5844B39BA6D0AC7E3B3CABA
2016-01-14 15:21:53,646 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:53,647 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:53,647 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:21:53,647 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:21:53,647 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:53,647 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:21:53,647 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:21:53,647 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D00171769664416C8ECB009AB2D6F4AD, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:21:53,647 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 2CA376B3B5844B39BA6D0AC7E3B3CABA, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:21:53,647 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:53,648 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 2CA376B3B5844B39BA6D0AC7E3B3CABA being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:53,648 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 2CA376B3B5844B39BA6D0AC7E3B3CABA
2016-01-14 15:21:53,648 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:53,648 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 2CA376B3B5844B39BA6D0AC7E3B3CABA
2016-01-14 15:21:53,648 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:21:53,648 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:53,648 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:53,648 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784913428_0001_1_00_000000_0
2016-01-14 15:21:53,649 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:21:53,649 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@5d795425
2016-01-14 15:21:53,651 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@5d795425
2016-01-14 15:21:53,651 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:21:53,651 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:21:53,651 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 15:21:53,653 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 2CA376B3B5844B39BA6D0AC7E3B3CABA
2016-01-14 15:21:53,653 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: 2A033B87DEC04D95AC7F6FDAC34D67E3
2016-01-14 15:21:53,653 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] 2A033B87DEC04D95AC7F6FDAC34D67E3
2016-01-14 15:21:53,654 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@7e06e077; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@757ae544; comparator=cascading.tuple.hadoop.util.TupleComparator@57620f9d; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:21:53,654 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:21:53,654 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:21:53,654 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:21:53,755 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:21:53,755 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:21:53,755 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:21:53,756 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:21:53,757 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:21:53,758 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 15:21:53,758 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:21:53,759 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D78EDD263AA34AD3A59647AA872D1C15, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:21:53,761 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784913428_0001_1_00_000000_0
2016-01-14 15:21:53,762 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D78EDD263AA34AD3A59647AA872D1C15, mem on close (mb), free: 177, total: 672, max: 672
2016-01-14 15:21:53,762 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:21:53,762 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:21:53,762 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 210; bufvoid = 104857600
2016-01-14 15:21:53,763 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214360(104857440); length = 37/6553600
2016-01-14 15:21:53,765 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:21:53 Completed running task attempt: attempt_1452784913428_0001_1_00_000000_0
2016-01-14 15:21:53,770 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784913428_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:53,771 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=704643072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452784913759, Process_Duration=2, Process_End_Time=1452784913761, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=10]]
2016-01-14 15:21:53,771 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784913428_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452784913428_0001_1_00, eventCount=1
2016-01-14 15:21:53,771 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784913428_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D78EDD263AA34AD3A59647AA872D1C15, taskAttemptId=attempt_1452784913428_0001_1_00_000000_0, startTime=1452784913622, finishTime=1452784913771, timeTaken=149, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=704643072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784913759, Process_Duration=2, Process_End_Time=1452784913761, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-01-14 15:21:53,771 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784913428_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:53,772 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784913428_0001_1_00_000000_0
2016-01-14 15:21:53,772 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:53,772 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:53,772 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784913428_0001_1][Event:TASK_FINISHED]: vertexName=D78EDD263AA34AD3A59647AA872D1C15, taskId=task_1452784913428_0001_1_00_000000, startTime=1452784913622, finishTime=1452784913772, timeTaken=150, status=SUCCEEDED, successfulAttemptID=attempt_1452784913428_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=704643072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784913759, Process_Duration=2, Process_End_Time=1452784913761, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-01-14 15:21:53,774 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:53,774 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:53,772 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784913428_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:53,774 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784913428_0001_1_00 [D78EDD263AA34AD3A59647AA872D1C15] : 1
2016-01-14 15:21:53,774 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784913428_0001_1_00 [D78EDD263AA34AD3A59647AA872D1C15]
2016-01-14 15:21:53,775 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:53,775 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784913428_0001_00_000001
2016-01-14 15:21:53,777 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:53,777 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784913428_0001_1][Event:VERTEX_FINISHED]: vertexName=D78EDD263AA34AD3A59647AA872D1C15, vertexId=vertex_1452784913428_0001_1_00, initRequestedTime=1452784913597, initedTime=1452784913609, startRequestedTime=1452784913610, startedTime=1452784913610, finishTime=1452784913774, timeTaken=164, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=704643072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784913759, Process_Duration=2, Process_End_Time=1452784913761, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1452784913622, firstTasksToStart=[ task_1452784913428_0001_1_00_000000 ], lastTaskFinishTime=1452784913771, lastTasksToFinish=[ task_1452784913428_0001_1_00_000000 ], minTaskDuration=149, maxTaskDuration=149, avgTaskDuration=149.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784913428_0001_1_00_000000 ], longestDurationTasks=[ task_1452784913428_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:53,777 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784913428_0001_1_00 [D78EDD263AA34AD3A59647AA872D1C15] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:53,777 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784913428_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:53,777 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784913428_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784913428_0001_00_000001, stoppedTime=1452784913777, exitStatus=0
2016-01-14 15:21:53,778 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784913428_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:53,778 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452784913428_0001_1_01 [D00171769664416C8ECB009AB2D6F4AD] attempt: attempt_1452784913428_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:21:53,778 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D00171769664416C8ECB009AB2D6F4AD
2016-01-14 15:21:53,778 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452784913428_0001_1_01 [D00171769664416C8ECB009AB2D6F4AD]
2016-01-14 15:21:53,778 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784913428_0001_1_00 [D78EDD263AA34AD3A59647AA872D1C15] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:21:53,777 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:53,778 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452784913428_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:21:53,778 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784913428_0001_1][Event:TASK_STARTED]: vertexName=D00171769664416C8ECB009AB2D6F4AD, taskId=task_1452784913428_0001_1_01_000000, scheduledTime=1452784913778, launchTime=1452784913778
2016-01-14 15:21:53,778 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784913428_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:21:53,779 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452784913428_0001_1_01 [D00171769664416C8ECB009AB2D6F4AD]
2016-01-14 15:21:53,779 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784913428_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:21:53,779 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:21:53,779 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [88A484A83144445F8032F24829241921/BFE2FCCF869A4FABB227C7E7D9814CFB] test/(1/1) ...latform/testSimple/simple, VertexName: D00171769664416C8ECB009AB2D6F4AD, VertexParallelism: 1, TaskAttemptID:attempt_1452784913428_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D78EDD263AA34AD3A59647AA872D1C15, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=1E54D205A69E413FA590B3BB7B4A5D86, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:53,779 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784913428_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:21:53,779 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:21:53,780 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452784913428_0001_00_000002
2016-01-14 15:21:53,780 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784913428_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:21:53,781 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452784913428_0001_00_000002
2016-01-14 15:21:53,781 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:21:53,783 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784913428_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:21:53,783 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:21:53,783 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784913428_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452784913428_0001_00_000002, launchTime=1452784913783
2016-01-14 15:21:53,784 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784913428_0001_00_000002
2016-01-14 15:21:53,784 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452784913428_0001_1_01_000000_0] to container: [container_1452784913428_0001_00_000002]
2016-01-14 15:21:53,784 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784913428_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:21:53,784 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452784913428_0001_00_000002 given task: attempt_1452784913428_0001_1_01_000000_0
2016-01-14 15:21:53,784 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784913428_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452784913428_0001_1_01_000000_0
2016-01-14 15:21:53,785 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452784913428_0001_1_01_000000_0] started. Is using containerId: [container_1452784913428_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:21:53,785 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784913428_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D00171769664416C8ECB009AB2D6F4AD, taskAttemptId=attempt_1452784913428_0001_1_01_000000_0, startTime=1452784913784, containerId=container_1452784913428_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452784913428_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:21:53,785 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784913428_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:21:53 Starting to run new task attempt: attempt_1452784913428_0001_1_01_000000_0
2016-01-14 15:21:53,785 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452784913428_0001_1_01_000000_0
2016-01-14 15:21:53,785 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784913428_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:21:53,785 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:53,785 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:21:53,786 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:21:53,785 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:21:53,786 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:21:53,786 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:21:53,786 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:21:53,786 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [88A484A83144445F8032F24829241921/BFE2FCCF869A4FABB227C7E7D9814CFB] test/(1/1) ...latform/testSimple/simple, VertexName: D00171769664416C8ECB009AB2D6F4AD, VertexParallelism: 1, TaskAttemptID:attempt_1452784913428_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D78EDD263AA34AD3A59647AA872D1C15, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=1E54D205A69E413FA590B3BB7B4A5D86, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:21:53,786 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:21:53,787 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452784913428_0001_1_01_000000_0
2016-01-14 15:21:53,787 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784913428_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:21:53,788 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784913428_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:53,788 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784913428_0001_00_000001, Exiting
2016-01-14 15:21:53,788 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784913428_0001_00_000001
2016-01-14 15:21:53,788 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784913428_0001_00_000001
2016-01-14 15:21:53,788 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452784913428_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:21:53,788 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:21:53,789 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:53,789 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784913428_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:53,789 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784913428_0001_00_000001 completed successfully
2016-01-14 15:21:53,790 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784913428_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:21:53,790 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D78EDD263AA34AD3A59647AA872D1C15, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:21:53,790 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=1E54D205A69E413FA590B3BB7B4A5D86, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:21:53,790 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:53,791 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:21:53,791 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:21:53,792 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D78EDD263AA34AD3A59647AA872D1C15
2016-01-14 15:21:53,792 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 1E54D205A69E413FA590B3BB7B4A5D86
2016-01-14 15:21:53,793 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:21:53,794 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 1E54D205A69E413FA590B3BB7B4A5D86
2016-01-14 15:21:53,793 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:21:53,795 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:21:53,794 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:21:53,795 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D78EDD263AA34AD3A59647AA872D1C15
2016-01-14 15:21:53,797 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D78EDD263AA34AD3A59647AA872D1C15
2016-01-14 15:21:53,803 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D00171769664416C8ECB009AB2D6F4AD, ordinal: 1
2016-01-14 15:21:53,804 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D00171769664416C8ECB009AB2D6F4AD, mem on start (mb), free: 173, total: 672, max: 672
2016-01-14 15:21:53,804 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:21:53,804 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:21:53,804 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:21:53,810 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:21:53,810 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 1E54D205A69E413FA590B3BB7B4A5D86
2016-01-14 15:21:53,810 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:21:53,811 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:21:53,811 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:21:53,811 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:21:53,811 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:21:53,812 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:21:53,812 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:21:53,812 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 1E54D205A69E413FA590B3BB7B4A5D86, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:21:53,812 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D78EDD263AA34AD3A59647AA872D1C15, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:21:53,813 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:21:53,813 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D78EDD263AA34AD3A59647AA872D1C15
2016-01-14 15:21:53,814 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:21:53,814 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:21:53,814 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D78EDD263AA34AD3A59647AA872D1C15 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:21:53,814 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D78EDD263AA34AD3A59647AA872D1C15 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:21:53,815 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:21:53,815 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:21:53,815 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:21:53,815 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D78EDD263AA34AD3A59647AA872D1C15
2016-01-14 15:21:53,816 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:21:53,816 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:21:53,817 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:21:53,817 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452784913428_0001_1_01_000000_0
2016-01-14 15:21:53,821 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452784913428_0001_1_00_000000_0_10088, runDuration: 127267, ]
2016-01-14 15:21:53,824 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-01-14 15:21:53,825 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: 2A033B87DEC04D95AC7F6FDAC34D67E3
2016-01-14 15:21:53,825 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple"], id: 1E54D205A69E413FA590B3BB7B4A5D86
2016-01-14 15:21:53,825 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple"]
2016-01-14 15:21:53,825 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] 2A033B87DEC04D95AC7F6FDAC34D67E3, for 1 inputs
2016-01-14 15:21:53,826 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452784913428_0001_1_00_000000_0_10088] to DISK_DIRECT, CompressedSize=204, DecompressedSize=200,EndTime=1452784913826, TimeTaken=4, Rate=0.05 MB/s
2016-01-14 15:21:53,826 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D78EDD263AA34AD3A59647AA872D1C15
2016-01-14 15:21:53,827 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:21:53,827 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [D78EDD263AA34AD3A59647AA872D1C15] #14 in 5ms
2016-01-14 15:21:53,827 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D78EDD263AA34AD3A59647AA872D1C15] #14, status:false, isInterrupted:false
2016-01-14 15:21:53,828 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:21:53,828 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 204 bytes from disk
2016-01-14 15:21:53,828 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:21:53,828 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:21:53,829 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 175 bytes
2016-01-14 15:21:53,829 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D78EDD263AA34AD3A59647AA872D1C15
2016-01-14 15:21:53,829 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:21:53,830 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D00171769664416C8ECB009AB2D6F4AD, all 1 inputs ready in: 00:00:00.005
2016-01-14 15:21:53,830 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@75087c8b; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@28c5b4
2016-01-14 15:21:53,836 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:21:53,836 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:21:53,836 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452784913428_0001_1_01_000000_0
2016-01-14 15:21:53,837 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452784913428_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:21:53,837 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452784913428_0001_1_01_000000_0
2016-01-14 15:21:53,838 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D00171769664416C8ECB009AB2D6F4AD, mem on close (mb), free: 170, total: 672, max: 672
2016-01-14 15:21:53,838 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D78EDD263AA34AD3A59647AA872D1C15
2016-01-14 15:21:53,838 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:21:53,839 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:21:53,839 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:21:53,839 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:21:53,839 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:21:53 Completed running task attempt: attempt_1452784913428_0001_1_01_000000_0
2016-01-14 15:21:53,841 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452784913428_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:21:53,842 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:21:53,842 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784913428_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D00171769664416C8ECB009AB2D6F4AD, taskAttemptId=attempt_1452784913428_0001_1_01_000000_0, startTime=1452784913784, finishTime=1452784913842, timeTaken=58, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=660, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=704643072, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784913830, Process_Duration=7, Process_End_Time=1452784913837, Read_Duration=1, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-01-14 15:21:53,842 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452784913428_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:21:53,843 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:21:53,842 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=660, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=704643072, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452784913830, Process_Duration=7, Process_End_Time=1452784913837, Read_Duration=1, Tuples_Read=10, Tuples_Written=8, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=8]]
2016-01-14 15:21:53,843 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452784913428_0001_1_01_000000_0
2016-01-14 15:21:53,843 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:21:53,844 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784913428_0001_1][Event:TASK_FINISHED]: vertexName=D00171769664416C8ECB009AB2D6F4AD, taskId=task_1452784913428_0001_1_01_000000, startTime=1452784913784, finishTime=1452784913843, timeTaken=59, status=SUCCEEDED, successfulAttemptID=attempt_1452784913428_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=660, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=704643072, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784913830, Process_Duration=7, Process_End_Time=1452784913837, Read_Duration=1, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-01-14 15:21:53,844 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452784913428_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:21:53,844 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452784913428_0001_1_01 [D00171769664416C8ECB009AB2D6F4AD] : 1
2016-01-14 15:21:53,844 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452784913428_0001_1_01 [D00171769664416C8ECB009AB2D6F4AD]
2016-01-14 15:21:53,844 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:21:53,845 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:53,845 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:21:53,845 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784913428_0001_1][Event:VERTEX_FINISHED]: vertexName=D00171769664416C8ECB009AB2D6F4AD, vertexId=vertex_1452784913428_0001_1_01, initRequestedTime=1452784913599, initedTime=1452784913607, startRequestedTime=1452784913613, startedTime=1452784913613, finishTime=1452784913844, timeTaken=231, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=660, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=704643072, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452784913830, Process_Duration=7, Process_End_Time=1452784913837, Read_Duration=1, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8, vertexStats=firstTaskStartTime=1452784913784, firstTasksToStart=[ task_1452784913428_0001_1_01_000000 ], lastTaskFinishTime=1452784913842, lastTasksToFinish=[ task_1452784913428_0001_1_01_000000 ], minTaskDuration=58, maxTaskDuration=58, avgTaskDuration=58.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452784913428_0001_1_01_000000 ], longestDurationTasks=[ task_1452784913428_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:21:53,845 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452784913428_0001_1_01 [D00171769664416C8ECB009AB2D6F4AD] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:21:53,846 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784913428_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:21:53,846 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784913428_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452784913428_0001_00_000002, stoppedTime=1452784913846, exitStatus=0
2016-01-14 15:21:53,846 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:21:53,846 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784913428_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:21:53,846 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452784913428_0001_1_01 [D00171769664416C8ECB009AB2D6F4AD] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:21:53,846 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452784913428_0001_1
2016-01-14 15:21:53,846 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784913428_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452784913428_0001_1
2016-01-14 15:21:53,846 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 1E54D205A69E413FA590B3BB7B4A5D86 for vertex: vertex_1452784913428_0001_1_01 [D00171769664416C8ECB009AB2D6F4AD]
2016-01-14 15:21:53,846 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452784913428_0001_00_000002
2016-01-14 15:21:53,851 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452784913428_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:21:53,851 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452784913428_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:21:53,852 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452784913428_0001_00_000002, Exiting
2016-01-14 15:21:53,852 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452784913428_0001_00_000002
2016-01-14 15:21:53,852 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452784913428_0001_00_000002
2016-01-14 15:21:53,852 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452784913428_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:21:53,852 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452784913428_0001_00_000002 completed successfully
2016-01-14 15:21:53,855 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452784913428_0001_1_00 [D78EDD263AA34AD3A59647AA872D1C15]
2016-01-14 15:21:53,856 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452784913428_0001_1, queueSize=0
2016-01-14 15:21:53,857 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:21:53,857 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452784913428_0001_1][Event:DAG_FINISHED]: dagId=dag_1452784913428_0001_1, startTime=1452784913597, finishTime=1452784913855, timeTaken=258, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2442, FILE_BYTES_WRITTEN=896, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1409286144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=18, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905569827589, Process_Duration=9, Process_End_Time=2905569827598, Read_Duration=1, Tuples_Read=20, Tuples_Written=18, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=8
2016-01-14 15:21:53,857 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452784913428_0001_1 finished with state: SUCCEEDED
2016-01-14 15:21:53,857 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452784913428_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:21:53,857 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452784913428_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:21:53 Completed Dag: dag_1452784913428_0001_1
2016-01-14 15:21:53,858 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452784913428_0001_1, dagState=SUCCEEDED
2016-01-14 15:21:53,858 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452784913428_0001_1_post
2016-01-14 15:21:53,858 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:21:53,858 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:21:53,858 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452784913428_0001_00_000002
2016-01-14 15:21:53,864 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testSimple/simple, applicationId=application_1452784913428_0001
2016-01-14 15:21:53,866 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:21:53,866 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:21:53,866 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:21:53,866 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:21:53,866 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:21:53,867 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:21:53,867 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:21:53,867 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:21:53,867 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:21:53,868 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:21:53,868 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:21:53,868 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:21:53,868 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:21:53,872 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/88A484A83144445F8032F24829241921/BFE2FCCF869A4FABB227C7E7D9814CFB/.tez/application_1452784913428_0001
2016-01-14 15:21:53,873 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:00.463
2016-01-14 15:21:53,889 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:21:53,889 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:21:53,905 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimple/simple