2016-01-14 16:29:41,318 INFO platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(195)) - found 1 test platforms from classpath
2016-01-14 16:29:41,342 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(244)) - adding test: cascading.BasicPipesPlatformTest, with platform: hadoop2-tez
2016-01-14 16:29:41,368 INFO tez.Hadoop2TezPlatform (Hadoop2TezPlatform.java:setUp(136)) - not using cluster
2016-01-14 16:29:41,695 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-01-14 16:29:42,001 INFO platform.TestPlatform (TestPlatform.java:getGlobalProperties(73)) - platform property overrides:
2016-01-14 16:29:42,170 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 16:29:42,172 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 16:29:42,254 INFO property.AppProps (AppProps.java:getAppID(177)) - using app.id: 430005C7B00A47A18105A13C83C4EBA9
2016-01-14 16:29:42,498 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.118
2016-01-14 16:29:42,529 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.150
2016-01-14 16:29:42,531 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-01-14 16:29:42,531 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-01-14 16:29:42,533 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 16:29:42,660 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-01-14 16:29:42,828 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain
2016-01-14 16:29:43,054 INFO util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 3.0.4
2016-01-14 16:29:43,058 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 16:29:43,059 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 16:29:43,059 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain"]
2016-01-14 16:29:43,059 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 16:29:43,060 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 16:29:43,060 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 16:29:43,061 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...stSimpleChain/simplechain
2016-01-14 16:29:43,061 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 16:29:43,079 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 16:29:43,232 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:43,238 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 16:29:43,239 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 16:29:43,281 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/28CA34D36F214374B6360543484EAC89/F52E247374D24B6983D348A84F56F557/.tez/application_1452788983238_0001 doesn't exist and is created
2016-01-14 16:29:43,342 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 16:29:43,342 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 16:29:43,344 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/28CA34D36F214374B6360543484EAC89/F52E247374D24B6983D348A84F56F557/.tez/application_1452788983238_0001
2016-01-14 16:29:43,374 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452788983238_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:43,414 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 16:29:43,429 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 16:29:43,443 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: NEW
2016-01-14 16:29:43,443 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(238)) - DAGAppMaster is not started wait for 100ms...
2016-01-14 16:29:43,443 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 16:29:43,454 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 16:29:43,457 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 16:29:43,461 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/28CA34D36F214374B6360543484EAC89/F52E247374D24B6983D348A84F56F557/.tez/application_1452788983238_0001/localmode-log-dir
2016-01-14 16:29:43,461 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/28CA34D36F214374B6360543484EAC89/F52E247374D24B6983D348A84F56F557/.tez/application_1452788983238_0001/localmode-log-dir/history.txt.appattempt_1452788983238_0001_000000, maxErrors=10
2016-01-14 16:29:43,462 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 16:29:43,464 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452788983238_0001_000000, appSubmitTime=1452788983360, launchTime=1452788983373
2016-01-14 16:29:43,473 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 16:29:43,474 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 16:29:43,484 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 16:29:43,484 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 16:29:43,538 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:41251
2016-01-14 16:29:43,543 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-01-14 16:29:43,544 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-01-14 16:29:43,544 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 16:29:43,581 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 16:29:43,644 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-01-14 16:29:43,644 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-01-14 16:29:43,645 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 16:29:43,652 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 16:29:43,652 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452788983238_0001_000000, startTime=1452788983652
2016-01-14 16:29:43,653 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 16:29:43,745 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 16:29:43,762 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 16:29:43,762 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...stSimpleChain/simplechain, applicationId=application_1452788983238_0001, dagName=[28CA34D36F214374B6360543484EAC89/F52E247374D24B6983D348A84F56F557] test/(1/1) ...stSimpleChain/simplechain
2016-01-14 16:29:43,980 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [28CA34D36F214374B6360543484EAC89/F52E247374D24B6983D348A84F56F557] test/(1/1) ...stSimpleChain/simplechain
2016-01-14 16:29:44,086 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452788983238_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/28CA34D36F214374B6360543484EAC89/F52E247374D24B6983D348A84F56F557/.tez/application_1452788983238_0001/localmode-log-dir/dag_1452788983238_0001_1.dot
2016-01-14 16:29:44,091 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/28CA34D36F214374B6360543484EAC89/F52E247374D24B6983D348A84F56F557/.tez/application_1452788983238_0001/localmode-log-dir/dag_1452788983238_0001_1-tez-dag.pb.txt
2016-01-14 16:29:44,192 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788983238_0001_1
2016-01-14 16:29:44,192 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:44,193 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [28CA34D36F214374B6360543484EAC89/F52E247374D24B6983D348A84F56F557] test/(1/1) ...stSimpleChain/simplechain
2016-01-14 16:29:44 Running Dag: dag_1452788983238_0001_1
2016-01-14 16:29:44,222 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 16:29:44,222 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452788983238_0001_1, submitTime=1452788983981
2016-01-14 16:29:44,254 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C3791DF384554A49AFF341458343BD5D
2016-01-14 16:29:44,262 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 16:29:44,265 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 16:29:44,265 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452788983238_0001_1, initTime=1452788984223
2016-01-14 16:29:44,265 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788983238_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 16:29:44,268 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 16:29:44,268 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:DAG_STARTED]: dagID=dag_1452788983238_0001_1, startTime=1452788984267
2016-01-14 16:29:44,271 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 16:29:44,273 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788983238_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 16:29:44,273 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C931C3757A29439EACA6513FC1687F0B : {5252AFC758EE4C1D8944EE84C266ABF5={InputName=5252AFC758EE4C1D8944EE84C266ABF5}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 16:29:44,274 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 5252AFC758EE4C1D8944EE84C266ABF5, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 16:29:44,274 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452788983238_0001_1_00 [C931C3757A29439EACA6513FC1687F0B]
2016-01-14 16:29:44,281 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452788983238_0001_1_00 [C931C3757A29439EACA6513FC1687F0B]
2016-01-14 16:29:44,281 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452788983238_0001_1_00 [C931C3757A29439EACA6513FC1687F0B]
2016-01-14 16:29:44,281 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...stSimpleChain/simplechain, applicationId=application_1452788983238_0001, dagName=[28CA34D36F214374B6360543484EAC89/F52E247374D24B6983D348A84F56F557] test/(1/1) ...stSimpleChain/simplechain
2016-01-14 16:29:44,284 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452788983238_0001_1_00 [C931C3757A29439EACA6513FC1687F0B]. Starting root input initializers: 1
2016-01-14 16:29:44,290 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 5252AFC758EE4C1D8944EE84C266ABF5 on vertex vertex_1452788983238_0001_1_00 [C931C3757A29439EACA6513FC1687F0B]
2016-01-14 16:29:44,346 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 16:29:44,347 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788983238_0001_1_00 [C931C3757A29439EACA6513FC1687F0B] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 16:29:44,349 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 5252AFC758EE4C1D8944EE84C266ABF5 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 16:29:44,349 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452788983238_0001_1_01 [E8BDE937E2E44F2DAE3C769317080D17]
2016-01-14 16:29:44,350 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 16:29:44,357 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 16:29:44,378 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452788983238_0001, with dag id: dag_1452788983238_0001_1
2016-01-14 16:29:44,391 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 16:29:44,391 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452788983238_0001_1_01 [E8BDE937E2E44F2DAE3C769317080D17]
2016-01-14 16:29:44,405 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-01-14 16:29:44,407 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 5252AFC758EE4C1D8944EE84C266ABF5 on vertex vertex_1452788983238_0001_1_00 [C931C3757A29439EACA6513FC1687F0B]
2016-01-14 16:29:44,411 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452788983238_0001_1_01 [E8BDE937E2E44F2DAE3C769317080D17]
2016-01-14 16:29:44,415 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:44,415 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E8BDE937E2E44F2DAE3C769317080D17, vertexId=vertex_1452788983238_0001_1_01, initRequestedTime=1452788984347, initedTime=1452788984414, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 16:29:44,415 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788983238_0001_1_01 [E8BDE937E2E44F2DAE3C769317080D17] transitioned from NEW to INITED due to event V_INIT
2016-01-14 16:29:44,423 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788983238_0001_1_00 [C931C3757A29439EACA6513FC1687F0B] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:44,431 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {5252AFC758EE4C1D8944EE84C266ABF5=forAllWorkUnits=true, update=[1]}
2016-01-14 16:29:44,431 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452788983238_0001_1_00 [C931C3757A29439EACA6513FC1687F0B] parallelism set to 1
2016-01-14 16:29:44,432 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788983238_0001_1_00 [C931C3757A29439EACA6513FC1687F0B] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:44,441 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788983238_0001_1_00, eventCount=1
2016-01-14 16:29:44,442 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:44,442 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C931C3757A29439EACA6513FC1687F0B, vertexId=vertex_1452788983238_0001_1_00, initRequestedTime=1452788984273, initedTime=1452788984441, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 16:29:44,442 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788983238_0001_1_00 [C931C3757A29439EACA6513FC1687F0B] with distanceFromRoot: 0
2016-01-14 16:29:44,442 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788983238_0001_1_00 [C931C3757A29439EACA6513FC1687F0B] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 16:29:44,442 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452788983238_0001_1_02 [C3791DF384554A49AFF341458343BD5D]
2016-01-14 16:29:44,460 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 16:29:44,460 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452788983238_0001_1_02 [C3791DF384554A49AFF341458343BD5D]
2016-01-14 16:29:44,461 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452788983238_0001_1_02 [C3791DF384554A49AFF341458343BD5D]
2016-01-14 16:29:44,461 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452788983238_0001_1_02 [C3791DF384554A49AFF341458343BD5D]
2016-01-14 16:29:44,461 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=78EE812A16A24841A094475D2EE0B7A6, vertexId=vertex_1452788983238_0001_1_02 [C3791DF384554A49AFF341458343BD5D], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 16:29:44,464 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=78EE812A16A24841A094475D2EE0B7A6, vertexId=vertex_1452788983238_0001_1_02 [C3791DF384554A49AFF341458343BD5D]
2016-01-14 16:29:44,478 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=78EE812A16A24841A094475D2EE0B7A6, vertexName=C3791DF384554A49AFF341458343BD5D, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 16:29:44,489 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=78EE812A16A24841A094475D2EE0B7A6, vertexId=vertex_1452788983238_0001_1_02 [C3791DF384554A49AFF341458343BD5D]
2016-01-14 16:29:44,492 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:44,492 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C3791DF384554A49AFF341458343BD5D, vertexId=vertex_1452788983238_0001_1_02, initRequestedTime=1452788984442, initedTime=1452788984492, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 16:29:44,492 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788983238_0001_1_02 [C3791DF384554A49AFF341458343BD5D] transitioned from NEW to INITED due to event V_INIT
2016-01-14 16:29:44,494 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C931C3757A29439EACA6513FC1687F0B
2016-01-14 16:29:44,494 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788983238_0001_1_00 [C931C3757A29439EACA6513FC1687F0B]
2016-01-14 16:29:44,496 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:44,497 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788983238_0001_1_00, startRequestedTime=1452788984493, startedTime=1452788984493
2016-01-14 16:29:44,497 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788983238_0001_1_00 [C931C3757A29439EACA6513FC1687F0B] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:44,525 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:44,526 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:TASK_STARTED]: vertexName=C931C3757A29439EACA6513FC1687F0B, taskId=task_1452788983238_0001_1_00_000000, scheduledTime=1452788984525, launchTime=1452788984525
2016-01-14 16:29:44,526 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788983238_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:44,527 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452788983238_0001_1_00 for vertex: vertex_1452788983238_0001_1_01 [E8BDE937E2E44F2DAE3C769317080D17] numStartedSources: 1 numSources: 1
2016-01-14 16:29:44,527 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788983238_0001_1_01 [E8BDE937E2E44F2DAE3C769317080D17] with distanceFromRoot: 1
2016-01-14 16:29:44,529 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788983238_0001_1_00 [C931C3757A29439EACA6513FC1687F0B]
2016-01-14 16:29:44,532 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C931C3757A29439EACA6513FC1687F0B in vertex: E8BDE937E2E44F2DAE3C769317080D17
2016-01-14 16:29:44,534 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E8BDE937E2E44F2DAE3C769317080D17 with 1 source tasks and 1 pending tasks
2016-01-14 16:29:44,535 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:44,535 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788983238_0001_1_01, startRequestedTime=1452788984529, startedTime=1452788984529
2016-01-14 16:29:44,535 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788983238_0001_1_01 [E8BDE937E2E44F2DAE3C769317080D17] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:44,537 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [28CA34D36F214374B6360543484EAC89/F52E247374D24B6983D348A84F56F557] test/(1/1) ...stSimpleChain/simplechain, VertexName: C931C3757A29439EACA6513FC1687F0B, VertexParallelism: 1, TaskAttemptID:attempt_1452788983238_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5252AFC758EE4C1D8944EE84C266ABF5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E8BDE937E2E44F2DAE3C769317080D17, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 16:29:44,540 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788983238_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:44,540 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452788983238_0001_1_01 for vertex: vertex_1452788983238_0001_1_02 [C3791DF384554A49AFF341458343BD5D] numStartedSources: 1 numSources: 1
2016-01-14 16:29:44,541 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788983238_0001_1_02 [C3791DF384554A49AFF341458343BD5D] with distanceFromRoot: 2
2016-01-14 16:29:44,541 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:44,542 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E8BDE937E2E44F2DAE3C769317080D17 in vertex: C3791DF384554A49AFF341458343BD5D
2016-01-14 16:29:44,542 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: C3791DF384554A49AFF341458343BD5D with 1 source tasks and 1 pending tasks
2016-01-14 16:29:44,543 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:44,543 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788983238_0001_1_02, startRequestedTime=1452788984541, startedTime=1452788984541
2016-01-14 16:29:44,543 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788983238_0001_1_02 [C3791DF384554A49AFF341458343BD5D] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:44,584 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 16:29:44,587 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 16:29:44,587 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 16:29:44,591 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788983238_0001_00_000001
2016-01-14 16:29:44,591 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788983238_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:44,599 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788983238_0001_00_000001
2016-01-14 16:29:44,599 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 16:29:44,603 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788983238_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:44,604 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788983238_0001_00_000001, launchTime=1452788984603
2016-01-14 16:29:44,604 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:44,607 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788983238_0001_00_000001
2016-01-14 16:29:44,612 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788983238_0001_1_00_000000_0] to container: [container_1452788983238_0001_00_000001]
2016-01-14 16:29:44,613 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788983238_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:44,616 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788983238_0001_1_00_000000_0] started. Is using containerId: [container_1452788983238_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 16:29:44,617 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C931C3757A29439EACA6513FC1687F0B, taskAttemptId=attempt_1452788983238_0001_1_00_000000_0, startTime=1452788984615, containerId=container_1452788983238_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788983238_0001_00_000001/teamcity, completedLogs=
2016-01-14 16:29:44,617 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788983238_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:44,618 INFO counters.Limits (Limits.java:ensureInitialized(59)) - Counter limits initialized with parameters: GROUP_NAME_MAX=128, MAX_GROUPS=500, COUNTER_NAME_MAX=64, MAX_COUNTERS=1200
2016-01-14 16:29:44,619 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:44,620 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788983238_0001_00_000001 given task: attempt_1452788983238_0001_1_00_000000_0
2016-01-14 16:29:44,621 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788983238_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788983238_0001_1_00_000000_0
2016-01-14 16:29:44 Starting to run new task attempt: attempt_1452788983238_0001_1_00_000000_0
2016-01-14 16:29:44,622 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788983238_0001_1_00_000000_0
2016-01-14 16:29:44,622 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:44,626 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788983238_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:44,626 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:44,627 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:44,628 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:44,630 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:44,651 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:44,652 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [28CA34D36F214374B6360543484EAC89/F52E247374D24B6983D348A84F56F557] test/(1/1) ...stSimpleChain/simplechain, VertexName: C931C3757A29439EACA6513FC1687F0B, VertexParallelism: 1, TaskAttemptID:attempt_1452788983238_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5252AFC758EE4C1D8944EE84C266ABF5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E8BDE937E2E44F2DAE3C769317080D17, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 16:29:44,654 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:44,658 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788983238_0001_1_00_000000_0
2016-01-14 16:29:44,659 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:44,659 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:44,664 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788983238_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 16:29:44,667 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=5252AFC758EE4C1D8944EE84C266ABF5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 16:29:44,670 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:44,670 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:44,670 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E8BDE937E2E44F2DAE3C769317080D17, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 16:29:44,670 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 5252AFC758EE4C1D8944EE84C266ABF5
2016-01-14 16:29:44,673 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 5252AFC758EE4C1D8944EE84C266ABF5
2016-01-14 16:29:44,673 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:44,682 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E8BDE937E2E44F2DAE3C769317080D17
2016-01-14 16:29:44,693 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 16:29:44,693 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E8BDE937E2E44F2DAE3C769317080D17
2016-01-14 16:29:44,694 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E8BDE937E2E44F2DAE3C769317080D17
2016-01-14 16:29:44,698 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 16:29:44,696 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:44,699 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 5252AFC758EE4C1D8944EE84C266ABF5
2016-01-14 16:29:44,700 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 5252AFC758EE4C1D8944EE84C266ABF5
2016-01-14 16:29:44,711 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C931C3757A29439EACA6513FC1687F0B, ordinal: 0
2016-01-14 16:29:44,712 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C931C3757A29439EACA6513FC1687F0B, mem on start (mb), free: 230, total: 281, max: 672
2016-01-14 16:29:44,712 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:44,712 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:44,712 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:44,712 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:44,714 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:44,720 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 16:29:44,721 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:44,721 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 16:29:44,721 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 16:29:44,722 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 16:29:44,722 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 5252AFC758EE4C1D8944EE84C266ABF5, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 16:29:44,723 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E8BDE937E2E44F2DAE3C769317080D17, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 16:29:44,723 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:44,724 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 5252AFC758EE4C1D8944EE84C266ABF5 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:44,725 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 16:29:44,725 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 5252AFC758EE4C1D8944EE84C266ABF5
2016-01-14 16:29:44,725 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:44,726 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 5252AFC758EE4C1D8944EE84C266ABF5
2016-01-14 16:29:44,725 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:44,728 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:44,729 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788983238_0001_1_00_000000_0
2016-01-14 16:29:44,735 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 16:29:44,745 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7601e2f3
2016-01-14 16:29:44,761 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7601e2f3
2016-01-14 16:29:44,763 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 16:29:44,768 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 16:29:44,769 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 16:29:44,796 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 5252AFC758EE4C1D8944EE84C266ABF5
2016-01-14 16:29:44,796 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: E757623251DA45E08EF80A70AD259E73
2016-01-14 16:29:44,797 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] E757623251DA45E08EF80A70AD259E73
2016-01-14 16:29:44,813 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@355a8e36; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@4c2e9697; comparator=cascading.tuple.hadoop.util.TupleComparator@6a0ba94; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 16:29:44,816 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 16:29:44,818 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 16:29:44,818 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 16:29:44,934 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 16:29:44,935 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 16:29:44,935 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 16:29:44,935 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 16:29:44,935 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 16:29:44,941 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 16:29:44,941 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:44,943 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C931C3757A29439EACA6513FC1687F0B, all 1 inputs ready in: 00:00:00.001
2016-01-14 16:29:44,951 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788983238_0001_1_00_000000_0
2016-01-14 16:29:44,951 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C931C3757A29439EACA6513FC1687F0B, mem on close (mb), free: 119, total: 281, max: 672
2016-01-14 16:29:44,952 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 16:29:44,952 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 16:29:44,952 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 210; bufvoid = 104857600
2016-01-14 16:29:44,953 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214360(104857440); length = 37/6553600
2016-01-14 16:29:44,967 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 16:29:44 Completed running task attempt: attempt_1452788983238_0001_1_00_000000_0
2016-01-14 16:29:44,988 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788983238_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:44,989 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788983238_0001_1_00, eventCount=1
2016-01-14 16:29:44,992 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452788984944, Process_Duration=7, Process_End_Time=1452788984951, Read_Duration=3, Tuples_Read=10, Tuples_Written=10, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=10]]
2016-01-14 16:29:44,992 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:44,997 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C931C3757A29439EACA6513FC1687F0B, taskAttemptId=attempt_1452788983238_0001_1_00_000000_0, startTime=1452788984615, finishTime=1452788984990, timeTaken=375, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788984944, Process_Duration=7, Process_End_Time=1452788984951, Read_Duration=3, Tuples_Read=10, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-01-14 16:29:44,999 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:44,999 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:45,000 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788983238_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:45,000 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788983238_0001_00_000001
2016-01-14 16:29:45,001 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:45,003 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788983238_0001_1_00_000000_0
2016-01-14 16:29:45,005 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:45,009 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:TASK_FINISHED]: vertexName=C931C3757A29439EACA6513FC1687F0B, taskId=task_1452788983238_0001_1_00_000000, startTime=1452788984615, finishTime=1452788985004, timeTaken=389, status=SUCCEEDED, successfulAttemptID=attempt_1452788983238_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788984944, Process_Duration=7, Process_End_Time=1452788984951, Read_Duration=3, Tuples_Read=10, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-01-14 16:29:45,010 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788983238_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:45,011 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788983238_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:45,012 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788983238_0001_1_00 [C931C3757A29439EACA6513FC1687F0B] : 1
2016-01-14 16:29:45,012 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788983238_0001_1_00 [C931C3757A29439EACA6513FC1687F0B]
2016-01-14 16:29:45,021 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:45,031 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:VERTEX_FINISHED]: vertexName=C931C3757A29439EACA6513FC1687F0B, vertexId=vertex_1452788983238_0001_1_00, initRequestedTime=1452788984273, initedTime=1452788984441, startRequestedTime=1452788984493, startedTime=1452788984493, finishTime=1452788985013, timeTaken=520, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788984944, Process_Duration=7, Process_End_Time=1452788984951, Read_Duration=3, Tuples_Read=10, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1452788984615, firstTasksToStart=[ task_1452788983238_0001_1_00_000000 ], lastTaskFinishTime=1452788984990, lastTasksToFinish=[ task_1452788983238_0001_1_00_000000 ], minTaskDuration=375, maxTaskDuration=375, avgTaskDuration=375.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788983238_0001_1_00_000000 ], longestDurationTasks=[ task_1452788983238_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:45,031 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788983238_0001_1_00 [C931C3757A29439EACA6513FC1687F0B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:45,033 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:45,034 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452788983238_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 16:29:45,034 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788983238_0001_00_000001: 32 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 16:29:45,034 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452788983238_0001_00_000001, Exiting
2016-01-14 16:29:45,034 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788983238_0001_00_000001
2016-01-14 16:29:45,034 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788983238_0001_00_000001
2016-01-14 16:29:45,037 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788983238_0001_00_000001, stoppedTime=1452788985033, exitStatus=0
2016-01-14 16:29:45,038 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788983238_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:45,039 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452788983238_0001_1_01 [E8BDE937E2E44F2DAE3C769317080D17] attempt: attempt_1452788983238_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 16:29:45,039 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788983238_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 16:29:45,040 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452788983238_0001_00_000001 completed successfully
2016-01-14 16:29:45,040 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E8BDE937E2E44F2DAE3C769317080D17
2016-01-14 16:29:45,041 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788983238_0001_1_01 [E8BDE937E2E44F2DAE3C769317080D17]
2016-01-14 16:29:45,043 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788983238_0001_1_00 [C931C3757A29439EACA6513FC1687F0B] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 16:29:45,043 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452788983238_0001_00_000001
2016-01-14 16:29:45,043 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788983238_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 16:29:45,044 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:45,044 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:TASK_STARTED]: vertexName=E8BDE937E2E44F2DAE3C769317080D17, taskId=task_1452788983238_0001_1_01_000000, scheduledTime=1452788985043, launchTime=1452788985043
2016-01-14 16:29:45,044 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788983238_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:45,044 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788983238_0001_1_01 [E8BDE937E2E44F2DAE3C769317080D17]
2016-01-14 16:29:45,045 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [28CA34D36F214374B6360543484EAC89/F52E247374D24B6983D348A84F56F557] test/(1/1) ...stSimpleChain/simplechain, VertexName: E8BDE937E2E44F2DAE3C769317080D17, VertexParallelism: 1, TaskAttemptID:attempt_1452788983238_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C931C3757A29439EACA6513FC1687F0B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C3791DF384554A49AFF341458343BD5D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 16:29:45,045 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788983238_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:45,045 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:45,046 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788983238_0001_00_000002
2016-01-14 16:29:45,047 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788983238_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:45,050 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788983238_0001_00_000002
2016-01-14 16:29:45,051 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 16:29:45,053 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788983238_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:45,053 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:45,054 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788983238_0001_00_000002, launchTime=1452788985053
2016-01-14 16:29:45,054 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788983238_0001_00_000002
2016-01-14 16:29:45,055 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788983238_0001_1_01_000000_0] to container: [container_1452788983238_0001_00_000002]
2016-01-14 16:29:45,055 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788983238_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:45,056 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788983238_0001_1_01_000000_0] started. Is using containerId: [container_1452788983238_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 16:29:45,056 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E8BDE937E2E44F2DAE3C769317080D17, taskAttemptId=attempt_1452788983238_0001_1_01_000000_0, startTime=1452788985056, containerId=container_1452788983238_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788983238_0001_00_000002/teamcity, completedLogs=
2016-01-14 16:29:45,056 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788983238_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:45,056 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788983238_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:45,057 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:45,057 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788983238_0001_00_000002 given task: attempt_1452788983238_0001_1_01_000000_0
2016-01-14 16:29:45,057 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788983238_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788983238_0001_1_01_000000_0
2016-01-14 16:29:45 Starting to run new task attempt: attempt_1452788983238_0001_1_01_000000_0
2016-01-14 16:29:45,058 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788983238_0001_1_01_000000_0
2016-01-14 16:29:45,058 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:45,058 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:45,058 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:45,058 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:45,058 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:45,059 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:45,059 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [28CA34D36F214374B6360543484EAC89/F52E247374D24B6983D348A84F56F557] test/(1/1) ...stSimpleChain/simplechain, VertexName: E8BDE937E2E44F2DAE3C769317080D17, VertexParallelism: 1, TaskAttemptID:attempt_1452788983238_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C931C3757A29439EACA6513FC1687F0B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C3791DF384554A49AFF341458343BD5D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 16:29:45,059 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:45,059 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788983238_0001_1_01_000000_0
2016-01-14 16:29:45,065 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:45,065 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:45,065 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788983238_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 16:29:45,066 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:45,066 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C3791DF384554A49AFF341458343BD5D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 16:29:45,066 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:45,066 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C3791DF384554A49AFF341458343BD5D
2016-01-14 16:29:45,067 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C931C3757A29439EACA6513FC1687F0B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 16:29:45,067 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:45,070 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C931C3757A29439EACA6513FC1687F0B
2016-01-14 16:29:45,077 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:45,081 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 16:29:45,081 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C3791DF384554A49AFF341458343BD5D
2016-01-14 16:29:45,082 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C3791DF384554A49AFF341458343BD5D
2016-01-14 16:29:45,090 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 16:29:45,090 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 16:29:45,090 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C931C3757A29439EACA6513FC1687F0B
2016-01-14 16:29:45,091 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E8BDE937E2E44F2DAE3C769317080D17, ordinal: 1
2016-01-14 16:29:45,091 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E8BDE937E2E44F2DAE3C769317080D17, mem on start (mb), free: 107, total: 281, max: 672
2016-01-14 16:29:45,091 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C931C3757A29439EACA6513FC1687F0B
2016-01-14 16:29:45,091 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:45,093 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:45,093 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:45,094 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:45,094 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:45,095 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 16:29:45,095 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 16:29:45,096 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 16:29:45,096 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 24, TotalRequested: 739036352, TotalRequestedScaled: 3.69518176E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:1.05
2016-01-14 16:29:45,097 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C3791DF384554A49AFF341458343BD5D, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=66985083
2016-01-14 16:29:45,097 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C931C3757A29439EACA6513FC1687F0B, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=405125774
2016-01-14 16:29:45,097 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:45,098 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C931C3757A29439EACA6513FC1687F0B
2016-01-14 16:29:45,098 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C931C3757A29439EACA6513FC1687F0B being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:45,100 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 16:29:45,101 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:45,109 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 16:29:45,111 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 16:29:45,116 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C931C3757A29439EACA6513FC1687F0B with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 16:29:45,118 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=405125774. Updated to: ShuffleMem=405125774, postMergeMem: 0
2016-01-14 16:29:45,119 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=405125774, maxSingleShuffleLimit=101281440, mergeThreshold=364613184, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 16:29:45,123 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 16:29:45,128 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C931C3757A29439EACA6513FC1687F0B
2016-01-14 16:29:45,129 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:45,129 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788983238_0001_1_01_000000_0
2016-01-14 16:29:45,132 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452788983238_0001_1_00_000000_0_10003, runDuration: 287386, ]
2016-01-14 16:29:45,137 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-01-14 16:29:45,146 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452788983238_0001_1_00_000000_0_10003] to DISK_DIRECT, CompressedSize=204, DecompressedSize=200,EndTime=1452788985146, TimeTaken=3, Rate=0.06 MB/s
2016-01-14 16:29:45,146 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C931C3757A29439EACA6513FC1687F0B
2016-01-14 16:29:45,147 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 16:29:45,148 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: E757623251DA45E08EF80A70AD259E73
2016-01-14 16:29:45,149 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [C931C3757A29439EACA6513FC1687F0B] #1 in 6ms
2016-01-14 16:29:45,149 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: EC202A87007741569CBAC4E5207E7CBE
2016-01-14 16:29:45,149 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C931C3757A29439EACA6513FC1687F0B] #1, status:false, isInterrupted:false
2016-01-14 16:29:45,152 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 16:29:45,154 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 204 bytes from disk
2016-01-14 16:29:45,155 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 16:29:45,150 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] EC202A87007741569CBAC4E5207E7CBE
2016-01-14 16:29:45,162 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 16:29:45,164 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@63cdfefe; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@692008e2; comparator=cascading.tuple.hadoop.util.TupleComparator@36276359; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 16:29:45,164 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 16:29:45,165 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 16:29:45,166 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 16:29:45,172 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 175 bytes
2016-01-14 16:29:45,172 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C931C3757A29439EACA6513FC1687F0B
2016-01-14 16:29:45,236 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 16515068(66060272)
2016-01-14 16:29:45,237 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 63
2016-01-14 16:29:45,237 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 52848232
2016-01-14 16:29:45,237 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 66060288
2016-01-14 16:29:45,238 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 16515068; length = 4128768
2016-01-14 16:29:45,237 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 16:29:45,241 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] E757623251DA45E08EF80A70AD259E73, for 1 inputs
2016-01-14 16:29:45,242 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E8BDE937E2E44F2DAE3C769317080D17, all 1 inputs ready in: 00:00:00.000
2016-01-14 16:29:45,244 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@a22d6a7; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@753a7323
2016-01-14 16:29:45,254 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788983238_0001_1_01_000000_0
2016-01-14 16:29:45,254 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E8BDE937E2E44F2DAE3C769317080D17, mem on close (mb), free: 102, total: 345, max: 672
2016-01-14 16:29:45,255 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C931C3757A29439EACA6513FC1687F0B
2016-01-14 16:29:45,255 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 16:29:45,256 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 16:29:45,257 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 16:29:45,257 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 16:29:45,258 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 16:29:45,259 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 16:29:45,259 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 16:29:45,260 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 184; bufvoid = 66060288
2016-01-14 16:29:45,260 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 16515068(66060272); kvend = 16515040(66060160); length = 29/4128768
2016-01-14 16:29:45,263 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 16:29:45 Completed running task attempt: attempt_1452788983238_0001_1_01_000000_0
2016-01-14 16:29:45,269 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788983238_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:45,269 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788983238_0001_1_01, eventCount=1
2016-01-14 16:29:45,270 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:45,272 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 49 [[File System Counters FILE_BYTES_READ=268, FILE_BYTES_WRITTEN=242, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=37, MERGE_PHASE_TIME=61, FIRST_EVENT_RECEIVED=31, LAST_EVENT_RECEIVED=31][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452788985242, Process_Duration=12, Process_End_Time=1452788985254, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0]]
2016-01-14 16:29:45,273 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:45,273 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:45,273 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788983238_0001_00_000002
2016-01-14 16:29:45,274 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E8BDE937E2E44F2DAE3C769317080D17, taskAttemptId=attempt_1452788983238_0001_1_01_000000_0, startTime=1452788985056, finishTime=1452788985270, timeTaken=214, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 49, File System Counters, FILE_BYTES_READ=268, FILE_BYTES_WRITTEN=242, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=37, MERGE_PHASE_TIME=61, FIRST_EVENT_RECEIVED=31, LAST_EVENT_RECEIVED=31, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788985242, Process_Duration=12, Process_End_Time=1452788985254, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0
2016-01-14 16:29:45,275 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788983238_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:45,275 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788983238_0001_1_01_000000_0
2016-01-14 16:29:45,276 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:45,277 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:45,278 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:TASK_FINISHED]: vertexName=E8BDE937E2E44F2DAE3C769317080D17, taskId=task_1452788983238_0001_1_01_000000, startTime=1452788985056, finishTime=1452788985275, timeTaken=219, status=SUCCEEDED, successfulAttemptID=attempt_1452788983238_0001_1_01_000000_0, diagnostics=, counters=Counters: 49, File System Counters, FILE_BYTES_READ=268, FILE_BYTES_WRITTEN=242, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=37, MERGE_PHASE_TIME=61, FIRST_EVENT_RECEIVED=31, LAST_EVENT_RECEIVED=31, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788985242, Process_Duration=12, Process_End_Time=1452788985254, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0
2016-01-14 16:29:45,279 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788983238_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:45,283 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788983238_0001_1_01 [E8BDE937E2E44F2DAE3C769317080D17] : 1
2016-01-14 16:29:45,283 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788983238_0001_1_01 [E8BDE937E2E44F2DAE3C769317080D17]
2016-01-14 16:29:45,285 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:45,288 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:VERTEX_FINISHED]: vertexName=E8BDE937E2E44F2DAE3C769317080D17, vertexId=vertex_1452788983238_0001_1_01, initRequestedTime=1452788984347, initedTime=1452788984414, startRequestedTime=1452788984529, startedTime=1452788984529, finishTime=1452788985284, timeTaken=755, status=SUCCEEDED, diagnostics=, counters=Counters: 49, File System Counters, FILE_BYTES_READ=268, FILE_BYTES_WRITTEN=242, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=37, MERGE_PHASE_TIME=61, FIRST_EVENT_RECEIVED=31, LAST_EVENT_RECEIVED=31, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788985242, Process_Duration=12, Process_End_Time=1452788985254, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, vertexStats=firstTaskStartTime=1452788985056, firstTasksToStart=[ task_1452788983238_0001_1_01_000000 ], lastTaskFinishTime=1452788985270, lastTasksToFinish=[ task_1452788983238_0001_1_01_000000 ], minTaskDuration=214, maxTaskDuration=214, avgTaskDuration=214.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788983238_0001_1_01_000000 ], longestDurationTasks=[ task_1452788983238_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:45,288 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788983238_0001_1_01 [E8BDE937E2E44F2DAE3C769317080D17] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:45,289 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788983238_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:45,289 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:45,289 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788983238_0001_00_000002, stoppedTime=1452788985289, exitStatus=0
2016-01-14 16:29:45,291 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788983238_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:45,291 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452788983238_0001_1_02 [C3791DF384554A49AFF341458343BD5D] attempt: attempt_1452788983238_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 16:29:45,292 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: C3791DF384554A49AFF341458343BD5D
2016-01-14 16:29:45,292 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788983238_0001_1_02 [C3791DF384554A49AFF341458343BD5D]
2016-01-14 16:29:45,292 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788983238_0001_1_01 [E8BDE937E2E44F2DAE3C769317080D17] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 16:29:45,293 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452788983238_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 16:29:45,294 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:45,294 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:TASK_STARTED]: vertexName=C3791DF384554A49AFF341458343BD5D, taskId=task_1452788983238_0001_1_02_000000, scheduledTime=1452788985294, launchTime=1452788985294
2016-01-14 16:29:45,294 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788983238_0001_1_02_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:45,294 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788983238_0001_1_02 [C3791DF384554A49AFF341458343BD5D]
2016-01-14 16:29:45,295 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788983238_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 16:29:45,295 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [28CA34D36F214374B6360543484EAC89/F52E247374D24B6983D348A84F56F557] test/(1/1) ...stSimpleChain/simplechain, VertexName: C3791DF384554A49AFF341458343BD5D, VertexParallelism: 1, TaskAttemptID:attempt_1452788983238_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E8BDE937E2E44F2DAE3C769317080D17, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=78EE812A16A24841A094475D2EE0B7A6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:45,295 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788983238_0001_1_02_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:45,295 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:45,297 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788983238_0001_00_000003
2016-01-14 16:29:45,297 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788983238_0001_00_000003 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:45,298 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788983238_0001_00_000003
2016-01-14 16:29:45,298 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - TezTask metrics system already initialized!
2016-01-14 16:29:45,299 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788983238_0001_00_000003 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:45,299 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788983238_0001_00_000003, launchTime=1452788985299
2016-01-14 16:29:45,299 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:45,300 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788983238_0001_00_000003
2016-01-14 16:29:45,301 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788983238_0001_1_02_000000_0] to container: [container_1452788983238_0001_00_000003]
2016-01-14 16:29:45,301 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788983238_0001_00_000003 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:45,301 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788983238_0001_00_000003 given task: attempt_1452788983238_0001_1_02_000000_0
2016-01-14 16:29:45,302 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788983238_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788983238_0001_1_02_000000_0
2016-01-14 16:29:45 Starting to run new task attempt: attempt_1452788983238_0001_1_02_000000_0
2016-01-14 16:29:45,302 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788983238_0001_1_02_000000_0
2016-01-14 16:29:45,302 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:45,302 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:45,303 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:45,303 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:45,303 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:45,303 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:45,303 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [28CA34D36F214374B6360543484EAC89/F52E247374D24B6983D348A84F56F557] test/(1/1) ...stSimpleChain/simplechain, VertexName: C3791DF384554A49AFF341458343BD5D, VertexParallelism: 1, TaskAttemptID:attempt_1452788983238_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E8BDE937E2E44F2DAE3C769317080D17, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=78EE812A16A24841A094475D2EE0B7A6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:45,303 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788983238_0001_1_02_000000_0] started. Is using containerId: [container_1452788983238_0001_00_000003] on NM: [127.0.0.1:0]
2016-01-14 16:29:45,304 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:45,304 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:45,305 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788983238_0001_1_02_000000_0
2016-01-14 16:29:45,305 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C3791DF384554A49AFF341458343BD5D, taskAttemptId=attempt_1452788983238_0001_1_02_000000_0, startTime=1452788985302, containerId=container_1452788983238_0001_00_000003, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788983238_0001_00_000003/teamcity, completedLogs=
2016-01-14 16:29:45,305 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788983238_0001_1_02_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:45,305 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:45,305 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:45,305 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788983238_0001_1_02_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:45,306 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452788983238_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 16:29:45,307 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788983238_0001_00_000002: 32 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 16:29:45,307 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452788983238_0001_00_000002, Exiting
2016-01-14 16:29:45,307 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788983238_0001_00_000002
2016-01-14 16:29:45,307 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:45,307 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E8BDE937E2E44F2DAE3C769317080D17, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 16:29:45,308 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:45,308 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E8BDE937E2E44F2DAE3C769317080D17
2016-01-14 16:29:45,307 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788983238_0001_1_02_000000_0 sent events: (0-1)
2016-01-14 16:29:45,307 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788983238_0001_00_000002
2016-01-14 16:29:45,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=78EE812A16A24841A094475D2EE0B7A6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 16:29:45,312 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788983238_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 16:29:45,313 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452788983238_0001_00_000002 completed successfully
2016-01-14 16:29:45,313 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788983238_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 16:29:45,315 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:45,313 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:45,316 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 78EE812A16A24841A094475D2EE0B7A6
2016-01-14 16:29:45,316 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 16:29:45,316 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 78EE812A16A24841A094475D2EE0B7A6
2016-01-14 16:29:45,319 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 16:29:45,320 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 16:29:45,320 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: E8BDE937E2E44F2DAE3C769317080D17
2016-01-14 16:29:45,322 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E8BDE937E2E44F2DAE3C769317080D17
2016-01-14 16:29:45,330 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C3791DF384554A49AFF341458343BD5D, ordinal: 2
2016-01-14 16:29:45,330 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C3791DF384554A49AFF341458343BD5D, mem on start (mb), free: 96, total: 345, max: 672
2016-01-14 16:29:45,330 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:45,330 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:45,330 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:45,341 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 16:29:45,341 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 78EE812A16A24841A094475D2EE0B7A6
2016-01-14 16:29:45,342 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:45,342 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:45,342 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 16:29:45,342 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:45,342 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 16:29:45,343 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 16:29:45,343 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 16:29:45,343 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 78EE812A16A24841A094475D2EE0B7A6, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 16:29:45,343 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E8BDE937E2E44F2DAE3C769317080D17, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 16:29:45,344 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:45,344 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E8BDE937E2E44F2DAE3C769317080D17 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:45,344 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E8BDE937E2E44F2DAE3C769317080D17
2016-01-14 16:29:45,346 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 16:29:45,346 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 16:29:45,346 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E8BDE937E2E44F2DAE3C769317080D17 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 16:29:45,347 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 16:29:45,347 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 16:29:45,347 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 16:29:45,347 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:45,348 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 16:29:45,348 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E8BDE937E2E44F2DAE3C769317080D17
2016-01-14 16:29:45,348 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:45,349 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788983238_0001_1_02_000000_0
2016-01-14 16:29:45,353 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452788983238_0001_1_01_000000_0_10005, runDuration: 199437, ]
2016-01-14 16:29:45,354 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-01-14 16:29:45,356 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: EC202A87007741569CBAC4E5207E7CBE
2016-01-14 16:29:45,358 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain"], id: 78EE812A16A24841A094475D2EE0B7A6
2016-01-14 16:29:45,359 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452788983238_0001_1_01_000000_0_10005] to DISK_DIRECT, CompressedSize=210, DecompressedSize=206,EndTime=1452788985358, TimeTaken=4, Rate=0.05 MB/s
2016-01-14 16:29:45,359 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E8BDE937E2E44F2DAE3C769317080D17
2016-01-14 16:29:45,359 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain"]
2016-01-14 16:29:45,360 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] EC202A87007741569CBAC4E5207E7CBE, for 1 inputs
2016-01-14 16:29:45,360 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 16:29:45,361 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [E8BDE937E2E44F2DAE3C769317080D17] #2 in 7ms
2016-01-14 16:29:45,361 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E8BDE937E2E44F2DAE3C769317080D17] #2, status:false, isInterrupted:false
2016-01-14 16:29:45,363 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 16:29:45,364 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 210 bytes from disk
2016-01-14 16:29:45,364 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 16:29:45,364 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 16:29:45,365 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 181 bytes
2016-01-14 16:29:45,366 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E8BDE937E2E44F2DAE3C769317080D17
2016-01-14 16:29:45,367 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C3791DF384554A49AFF341458343BD5D, all 1 inputs ready in: 00:00:00.006
2016-01-14 16:29:45,367 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 16:29:45,368 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@608d1c76; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@5047b87f
2016-01-14 16:29:45,371 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 16:29:45,372 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 16:29:45,374 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452788983238_0001_1_02_000000_0
2016-01-14 16:29:45,374 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452788983238_0001_1_02_000000_0 given a go for committing the task output.
2016-01-14 16:29:45,375 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788983238_0001_1_02_000000_0
2016-01-14 16:29:45,375 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C3791DF384554A49AFF341458343BD5D, mem on close (mb), free: 92, total: 345, max: 672
2016-01-14 16:29:45,376 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E8BDE937E2E44F2DAE3C769317080D17
2016-01-14 16:29:45,377 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 16:29:45,377 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 16:29:45,377 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 16:29:45,378 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 16:29:45,378 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 16:29:45 Completed running task attempt: attempt_1452788983238_0001_1_02_000000_0
2016-01-14 16:29:45,380 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788983238_0001_1_02_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:45,382 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:45,383 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C3791DF384554A49AFF341458343BD5D, taskAttemptId=attempt_1452788983238_0001_1_02_000000_0, startTime=1452788985302, finishTime=1452788985382, timeTaken=80, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=242, FILE_BYTES_WRITTEN=660, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=15, MERGE_PHASE_TIME=20, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788985367, Process_Duration=8, Process_End_Time=1452788985375, Read_Duration=2, Tuples_Read=8, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-01-14 16:29:45,383 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788983238_0001_1_02_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:45,384 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:45,384 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788983238_0001_1_02_000000_0
2016-01-14 16:29:45,385 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:45,388 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:TASK_FINISHED]: vertexName=C3791DF384554A49AFF341458343BD5D, taskId=task_1452788983238_0001_1_02_000000, startTime=1452788985302, finishTime=1452788985384, timeTaken=82, status=SUCCEEDED, successfulAttemptID=attempt_1452788983238_0001_1_02_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=242, FILE_BYTES_WRITTEN=660, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=15, MERGE_PHASE_TIME=20, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788985367, Process_Duration=8, Process_End_Time=1452788985375, Read_Duration=2, Tuples_Read=8, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-01-14 16:29:45,389 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788983238_0001_1_02_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:45,390 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788983238_0001_1_02 [C3791DF384554A49AFF341458343BD5D] : 1
2016-01-14 16:29:45,390 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788983238_0001_1_02 [C3791DF384554A49AFF341458343BD5D]
2016-01-14 16:29:45,391 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:45,392 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=242, FILE_BYTES_WRITTEN=660, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=15, MERGE_PHASE_TIME=20, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452788985367, Process_Duration=8, Process_End_Time=1452788985375, Read_Duration=2, Tuples_Read=8, Tuples_Written=8, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=8]]
2016-01-14 16:29:45,393 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:45,393 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:45,394 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788983238_0001_00_000003
2016-01-14 16:29:45,396 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:VERTEX_FINISHED]: vertexName=C3791DF384554A49AFF341458343BD5D, vertexId=vertex_1452788983238_0001_1_02, initRequestedTime=1452788984442, initedTime=1452788984492, startRequestedTime=1452788984541, startedTime=1452788984541, finishTime=1452788985390, timeTaken=849, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=242, FILE_BYTES_WRITTEN=660, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=15, MERGE_PHASE_TIME=20, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788985367, Process_Duration=8, Process_End_Time=1452788985375, Read_Duration=2, Tuples_Read=8, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8, vertexStats=firstTaskStartTime=1452788985302, firstTasksToStart=[ task_1452788983238_0001_1_02_000000 ], lastTaskFinishTime=1452788985382, lastTasksToFinish=[ task_1452788983238_0001_1_02_000000 ], minTaskDuration=80, maxTaskDuration=80, avgTaskDuration=80.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788983238_0001_1_02_000000 ], longestDurationTasks=[ task_1452788983238_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:45,396 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788983238_0001_1_02 [C3791DF384554A49AFF341458343BD5D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:45,397 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788983238_0001_00_000003 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:45,397 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:45,398 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788983238_0001_00_000003, stoppedTime=1452788985397, exitStatus=0
2016-01-14 16:29:45,398 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788983238_0001_00_000003 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:45,399 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788983238_0001_1_02 [C3791DF384554A49AFF341458343BD5D] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 16:29:45,399 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452788983238_0001_1
2016-01-14 16:29:45,401 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452788983238_0001_1
2016-01-14 16:29:45,402 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 78EE812A16A24841A094475D2EE0B7A6 for vertex: vertex_1452788983238_0001_1_02 [C3791DF384554A49AFF341458343BD5D]
2016-01-14 16:29:45,406 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452788983238_0001_00_000003 is valid, but no longer registered, and will be killed
2016-01-14 16:29:45,406 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788983238_0001_00_000003: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 16:29:45,406 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452788983238_0001_00_000003, Exiting
2016-01-14 16:29:45,406 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788983238_0001_00_000003
2016-01-14 16:29:45,407 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788983238_0001_00_000003
2016-01-14 16:29:45,407 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788983238_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 16:29:45,407 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452788983238_0001_00_000003 completed successfully
2016-01-14 16:29:45,414 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452788983238_0001_1_01 [E8BDE937E2E44F2DAE3C769317080D17]
2016-01-14 16:29:45,415 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452788983238_0001_1_00 [C931C3757A29439EACA6513FC1687F0B]
2016-01-14 16:29:45,422 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452788983238_0001_1, queueSize=0
2016-01-14 16:29:45,423 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 16:29:45,425 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788983238_0001_1][Event:DAG_FINISHED]: dagId=dag_1452788983238_0001_1, startTime=1452788984267, finishTime=1452788985415, timeTaken=1148, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2716, FILE_BYTES_WRITTEN=1138, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=14, REDUCE_INPUT_RECORDS=18, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=36, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1018691584, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=26, OUTPUT_BYTES=394, OUTPUT_BYTES_WITH_OVERHEAD=406, OUTPUT_BYTES_PHYSICAL=414, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=414, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=414, SHUFFLE_BYTES_DECOMPRESSED=406, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=414, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=52, MERGE_PHASE_TIME=81, FIRST_EVENT_RECEIVED=39, LAST_EVENT_RECEIVED=39, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4358366955553, Process_Duration=27, Process_End_Time=4358366955580, Read_Duration=7, Tuples_Read=28, Tuples_Written=26, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=8
2016-01-14 16:29:45,426 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452788983238_0001_1 finished with state: SUCCEEDED
2016-01-14 16:29:45,426 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788983238_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 16:29:45,426 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452788983238_0001_00_000003
2016-01-14 16:29:45,427 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788983238_0001_00_000003 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 16:29:45 Completed Dag: dag_1452788983238_0001_1
2016-01-14 16:29:45,427 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452788983238_0001_1, dagState=SUCCEEDED
2016-01-14 16:29:45,427 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788983238_0001_1_post
2016-01-14 16:29:45,428 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:45,428 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 16:29:45,437 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...stSimpleChain/simplechain, applicationId=application_1452788983238_0001
2016-01-14 16:29:45,441 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 16:29:45,441 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 16:29:45,442 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 16:29:45,442 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 16:29:45,442 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 16:29:45,444 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 16:29:45,445 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 16:29:45,445 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 16:29:45,446 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 16:29:45,446 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 16:29:45,446 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 16:29:45,446 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 16:29:45,447 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 16:29:45,454 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/28CA34D36F214374B6360543484EAC89/F52E247374D24B6983D348A84F56F557/.tez/application_1452788983238_0001
2016-01-14 16:29:45,454 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:02.396
2016-01-14 16:29:45,502 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain
2016-01-14 16:29:45,539 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 16:29:45,539 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 16:29:45,601 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.028
2016-01-14 16:29:45,637 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.064
2016-01-14 16:29:45,639 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 16:29:45,639 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 16:29:45,639 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 16:29:45,661 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-01-14 16:29:45,677 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count
2016-01-14 16:29:45,794 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] starting
2016-01-14 16:29:45,794 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 16:29:45,795 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count"]
2016-01-14 16:29:45,795 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] parallel execution of steps is enabled: false
2016-01-14 16:29:45,795 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executing total steps: 1
2016-01-14 16:29:45,796 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] allocating management threads: 1
2016-01-14 16:29:45,796 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] starting step: (1/1) ...splatform/testCount/count
2016-01-14 16:29:45,797 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 16:29:45,817 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tez session mode enabled: true
2016-01-14 16:29:45,824 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:45,824 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 16:29:45,824 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 16:29:45,837 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0D68C3D7098245DFBFF00D5C923580BB/EDCAD390A7A346A69DAD890D3B3035C1/.tez/application_1452788985824_0001 doesn't exist and is created
2016-01-14 16:29:45,859 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 16:29:45,859 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 16:29:45,860 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0D68C3D7098245DFBFF00D5C923580BB/EDCAD390A7A346A69DAD890D3B3035C1/.tez/application_1452788985824_0001
2016-01-14 16:29:45,865 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452788985824_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:45,879 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 16:29:45,880 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 16:29:45,883 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 16:29:45,884 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 16:29:45,884 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0D68C3D7098245DFBFF00D5C923580BB/EDCAD390A7A346A69DAD890D3B3035C1/.tez/application_1452788985824_0001/localmode-log-dir
2016-01-14 16:29:45,884 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0D68C3D7098245DFBFF00D5C923580BB/EDCAD390A7A346A69DAD890D3B3035C1/.tez/application_1452788985824_0001/localmode-log-dir/history.txt.appattempt_1452788985824_0001_000000, maxErrors=10
2016-01-14 16:29:45,884 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 16:29:45,885 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452788985824_0001_000000, appSubmitTime=1452788985864, launchTime=1452788985864
2016-01-14 16:29:45,891 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 16:29:45,896 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 16:29:45,896 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:46998
2016-01-14 16:29:45,900 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 16:29:45,900 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 16:29:45,904 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 16:29:45,907 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 16:29:45,907 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452788985824_0001_000000, startTime=1452788985907
2016-01-14 16:29:45,907 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 16:29:45,960 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 16:29:45,960 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 16:29:45,960 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...splatform/testCount/count, applicationId=application_1452788985824_0001, dagName=[0D68C3D7098245DFBFF00D5C923580BB/EDCAD390A7A346A69DAD890D3B3035C1] count/(1/1) ...splatform/testCount/count
2016-01-14 16:29:45,980 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [0D68C3D7098245DFBFF00D5C923580BB/EDCAD390A7A346A69DAD890D3B3035C1] count/(1/1) ...splatform/testCount/count
2016-01-14 16:29:45,983 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452788985824_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0D68C3D7098245DFBFF00D5C923580BB/EDCAD390A7A346A69DAD890D3B3035C1/.tez/application_1452788985824_0001/localmode-log-dir/dag_1452788985824_0001_1.dot
2016-01-14 16:29:45,984 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0D68C3D7098245DFBFF00D5C923580BB/EDCAD390A7A346A69DAD890D3B3035C1/.tez/application_1452788985824_0001/localmode-log-dir/dag_1452788985824_0001_1-tez-dag.pb.txt
2016-01-14 16:29:46,012 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788985824_0001_1
2016-01-14 16:29:46,013 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:46,013 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [0D68C3D7098245DFBFF00D5C923580BB/EDCAD390A7A346A69DAD890D3B3035C1] count/(1/1) ...splatform/testCount/count
2016-01-14 16:29:46 Running Dag: dag_1452788985824_0001_1
2016-01-14 16:29:46,030 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788985824_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452788985824_0001_1, submitTime=1452788985980
2016-01-14 16:29:46,030 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 16:29:46,031 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C84E2E16ED6E44ABB4C6D3FC63AAD093
2016-01-14 16:29:46,031 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 16:29:46,032 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 16:29:46,032 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788985824_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452788985824_0001_1, initTime=1452788986030
2016-01-14 16:29:46,032 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788985824_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 16:29:46,034 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 16:29:46,034 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788985824_0001_1][Event:DAG_STARTED]: dagID=dag_1452788985824_0001_1, startTime=1452788986033
2016-01-14 16:29:46,034 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 16:29:46,034 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788985824_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 16:29:46,034 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A9FD6DD5CCEC4C298033735B61E6F580 : {C4677FD2402D46EB83CB643CFC62577D={InputName=C4677FD2402D46EB83CB643CFC62577D}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 16:29:46,035 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: C4677FD2402D46EB83CB643CFC62577D, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 16:29:46,035 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452788985824_0001_1_00 [A9FD6DD5CCEC4C298033735B61E6F580]
2016-01-14 16:29:46,035 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452788985824_0001_1_00 [A9FD6DD5CCEC4C298033735B61E6F580]
2016-01-14 16:29:46,035 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452788985824_0001_1_00 [A9FD6DD5CCEC4C298033735B61E6F580]
2016-01-14 16:29:46,035 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...splatform/testCount/count, applicationId=application_1452788985824_0001, dagName=[0D68C3D7098245DFBFF00D5C923580BB/EDCAD390A7A346A69DAD890D3B3035C1] count/(1/1) ...splatform/testCount/count
2016-01-14 16:29:46,036 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452788985824_0001_1_00 [A9FD6DD5CCEC4C298033735B61E6F580]. Starting root input initializers: 1
2016-01-14 16:29:46,036 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788985824_0001_1_00 [A9FD6DD5CCEC4C298033735B61E6F580] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 16:29:46,036 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: C4677FD2402D46EB83CB643CFC62577D on vertex vertex_1452788985824_0001_1_00 [A9FD6DD5CCEC4C298033735B61E6F580]
2016-01-14 16:29:46,036 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452788985824_0001_1_01 [C84E2E16ED6E44ABB4C6D3FC63AAD093]
2016-01-14 16:29:46,040 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 16:29:46,041 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input C4677FD2402D46EB83CB643CFC62577D asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 16:29:46,041 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 16:29:46,042 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452788985824_0001_1_01 [C84E2E16ED6E44ABB4C6D3FC63AAD093]
2016-01-14 16:29:46,043 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452788985824_0001_1_01 [C84E2E16ED6E44ABB4C6D3FC63AAD093]
2016-01-14 16:29:46,044 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452788985824_0001_1_01 [C84E2E16ED6E44ABB4C6D3FC63AAD093]
2016-01-14 16:29:46,044 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=74CE8B0ABE1A40B3AF6875113FD0E97B, vertexId=vertex_1452788985824_0001_1_01 [C84E2E16ED6E44ABB4C6D3FC63AAD093], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 16:29:46,045 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=74CE8B0ABE1A40B3AF6875113FD0E97B, vertexId=vertex_1452788985824_0001_1_01 [C84E2E16ED6E44ABB4C6D3FC63AAD093]
2016-01-14 16:29:46,044 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 16:29:46,045 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 16:29:46,048 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-01-14 16:29:46,049 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=74CE8B0ABE1A40B3AF6875113FD0E97B, vertexName=C84E2E16ED6E44ABB4C6D3FC63AAD093, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 16:29:46,050 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=74CE8B0ABE1A40B3AF6875113FD0E97B, vertexId=vertex_1452788985824_0001_1_01 [C84E2E16ED6E44ABB4C6D3FC63AAD093]
2016-01-14 16:29:46,049 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: C4677FD2402D46EB83CB643CFC62577D on vertex vertex_1452788985824_0001_1_00 [A9FD6DD5CCEC4C298033735B61E6F580]
2016-01-14 16:29:46,052 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:46,052 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788985824_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C84E2E16ED6E44ABB4C6D3FC63AAD093, vertexId=vertex_1452788985824_0001_1_01, initRequestedTime=1452788986036, initedTime=1452788986052, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 16:29:46,053 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788985824_0001_1_01 [C84E2E16ED6E44ABB4C6D3FC63AAD093] transitioned from NEW to INITED due to event V_INIT
2016-01-14 16:29:46,053 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788985824_0001_1_00 [A9FD6DD5CCEC4C298033735B61E6F580] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:46,054 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {C4677FD2402D46EB83CB643CFC62577D=forAllWorkUnits=true, update=[1]}
2016-01-14 16:29:46,054 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452788985824_0001_1_00 [A9FD6DD5CCEC4C298033735B61E6F580] parallelism set to 1
2016-01-14 16:29:46,055 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788985824_0001_1_00 [A9FD6DD5CCEC4C298033735B61E6F580] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:46,055 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788985824_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788985824_0001_1_00, eventCount=1
2016-01-14 16:29:46,056 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:46,057 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] submitted tez dag to app master: application_1452788985824_0001, with dag id: dag_1452788985824_0001_1
2016-01-14 16:29:46,057 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788985824_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A9FD6DD5CCEC4C298033735B61E6F580, vertexId=vertex_1452788985824_0001_1_00, initRequestedTime=1452788986034, initedTime=1452788986056, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 16:29:46,059 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788985824_0001_1_00 [A9FD6DD5CCEC4C298033735B61E6F580] with distanceFromRoot: 0
2016-01-14 16:29:46,059 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788985824_0001_1_00 [A9FD6DD5CCEC4C298033735B61E6F580] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 16:29:46,060 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A9FD6DD5CCEC4C298033735B61E6F580
2016-01-14 16:29:46,060 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788985824_0001_1_00 [A9FD6DD5CCEC4C298033735B61E6F580]
2016-01-14 16:29:46,060 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:46,061 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788985824_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788985824_0001_1_00, startRequestedTime=1452788986059, startedTime=1452788986059
2016-01-14 16:29:46,061 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788985824_0001_1_00 [A9FD6DD5CCEC4C298033735B61E6F580] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:46,066 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:46,066 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788985824_0001_1][Event:TASK_STARTED]: vertexName=A9FD6DD5CCEC4C298033735B61E6F580, taskId=task_1452788985824_0001_1_00_000000, scheduledTime=1452788986064, launchTime=1452788986064
2016-01-14 16:29:46,067 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788985824_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:46,067 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452788985824_0001_1_00 for vertex: vertex_1452788985824_0001_1_01 [C84E2E16ED6E44ABB4C6D3FC63AAD093] numStartedSources: 1 numSources: 1
2016-01-14 16:29:46,069 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788985824_0001_1_01 [C84E2E16ED6E44ABB4C6D3FC63AAD093] with distanceFromRoot: 1
2016-01-14 16:29:46,070 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788985824_0001_1_00 [A9FD6DD5CCEC4C298033735B61E6F580]
2016-01-14 16:29:46,070 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: A9FD6DD5CCEC4C298033735B61E6F580 in vertex: C84E2E16ED6E44ABB4C6D3FC63AAD093
2016-01-14 16:29:46,070 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: C84E2E16ED6E44ABB4C6D3FC63AAD093 with 1 source tasks and 1 pending tasks
2016-01-14 16:29:46,071 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:46,071 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788985824_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788985824_0001_1_01, startRequestedTime=1452788986070, startedTime=1452788986070
2016-01-14 16:29:46,072 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788985824_0001_1_01 [C84E2E16ED6E44ABB4C6D3FC63AAD093] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:46,076 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [0D68C3D7098245DFBFF00D5C923580BB/EDCAD390A7A346A69DAD890D3B3035C1] count/(1/1) ...splatform/testCount/count, VertexName: A9FD6DD5CCEC4C298033735B61E6F580, VertexParallelism: 1, TaskAttemptID:attempt_1452788985824_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C4677FD2402D46EB83CB643CFC62577D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C84E2E16ED6E44ABB4C6D3FC63AAD093, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 16:29:46,078 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788985824_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:46,080 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:46,083 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 16:29:46,084 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 16:29:46,084 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 16:29:46,085 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788985824_0001_00_000001
2016-01-14 16:29:46,085 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788985824_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:46,088 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788985824_0001_00_000001
2016-01-14 16:29:46,088 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 16:29:46,089 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788985824_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:46,089 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:46,090 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788985824_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788985824_0001_00_000001, launchTime=1452788986089
2016-01-14 16:29:46,090 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788985824_0001_00_000001
2016-01-14 16:29:46,091 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788985824_0001_1_00_000000_0] to container: [container_1452788985824_0001_00_000001]
2016-01-14 16:29:46,091 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788985824_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:46,092 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788985824_0001_00_000001 given task: attempt_1452788985824_0001_1_00_000000_0
2016-01-14 16:29:46,092 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788985824_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788985824_0001_1_00_000000_0
2016-01-14 16:29:46,092 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788985824_0001_1_00_000000_0] started. Is using containerId: [container_1452788985824_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 16:29:46,093 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788985824_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A9FD6DD5CCEC4C298033735B61E6F580, taskAttemptId=attempt_1452788985824_0001_1_00_000000_0, startTime=1452788986092, containerId=container_1452788985824_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788985824_0001_00_000001/teamcity, completedLogs=
2016-01-14 16:29:46 Starting to run new task attempt: attempt_1452788985824_0001_1_00_000000_0
2016-01-14 16:29:46,093 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788985824_0001_1_00_000000_0
2016-01-14 16:29:46,093 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:46,093 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:46,093 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788985824_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:46,094 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788985824_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:46,094 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:46,094 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:46,095 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:46,095 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:46,095 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:46,095 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [0D68C3D7098245DFBFF00D5C923580BB/EDCAD390A7A346A69DAD890D3B3035C1] count/(1/1) ...splatform/testCount/count, VertexName: A9FD6DD5CCEC4C298033735B61E6F580, VertexParallelism: 1, TaskAttemptID:attempt_1452788985824_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C4677FD2402D46EB83CB643CFC62577D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C84E2E16ED6E44ABB4C6D3FC63AAD093, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 16:29:46,096 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:46,098 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788985824_0001_1_00_000000_0
2016-01-14 16:29:46,103 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:46,104 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:46,106 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:46,106 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788985824_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 16:29:46,107 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C4677FD2402D46EB83CB643CFC62577D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 16:29:46,107 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:46,107 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C4677FD2402D46EB83CB643CFC62577D
2016-01-14 16:29:46,107 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: C4677FD2402D46EB83CB643CFC62577D
2016-01-14 16:29:46,111 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:46,114 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C84E2E16ED6E44ABB4C6D3FC63AAD093, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 16:29:46,118 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:46,118 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C84E2E16ED6E44ABB4C6D3FC63AAD093
2016-01-14 16:29:46,114 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 16:29:46,119 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: C4677FD2402D46EB83CB643CFC62577D
2016-01-14 16:29:46,119 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C4677FD2402D46EB83CB643CFC62577D
2016-01-14 16:29:46,120 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 16:29:46,120 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C84E2E16ED6E44ABB4C6D3FC63AAD093
2016-01-14 16:29:46,121 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C84E2E16ED6E44ABB4C6D3FC63AAD093
2016-01-14 16:29:46,126 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A9FD6DD5CCEC4C298033735B61E6F580, ordinal: 0
2016-01-14 16:29:46,126 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A9FD6DD5CCEC4C298033735B61E6F580, mem on start (mb), free: 216, total: 461, max: 672
2016-01-14 16:29:46,126 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:46,127 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:46,127 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:46,127 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:46,128 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:46,128 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 16:29:46,128 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:46,129 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 16:29:46,129 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 16:29:46,130 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 16:29:46,130 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C4677FD2402D46EB83CB643CFC62577D, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 16:29:46,130 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C84E2E16ED6E44ABB4C6D3FC63AAD093, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 16:29:46,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:46,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C4677FD2402D46EB83CB643CFC62577D
2016-01-14 16:29:46,131 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:46,131 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C4677FD2402D46EB83CB643CFC62577D
2016-01-14 16:29:46,131 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C4677FD2402D46EB83CB643CFC62577D being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:46,131 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 16:29:46,132 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:46,132 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:46,133 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788985824_0001_1_00_000000_0
2016-01-14 16:29:46,133 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 16:29:46,135 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1e541ea
2016-01-14 16:29:46,136 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1e541ea
2016-01-14 16:29:46,137 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 16:29:46,137 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 16:29:46,140 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 16:29:46,140 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: C4677FD2402D46EB83CB643CFC62577D
2016-01-14 16:29:46,140 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: C771EE2E0FE043299DF05B81A0B20ABB
2016-01-14 16:29:46,141 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] C771EE2E0FE043299DF05B81A0B20ABB
2016-01-14 16:29:46,142 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@1ee5c7ab; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@7f97bb65; comparator=cascading.tuple.hadoop.util.TupleComparator@42902554; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 16:29:46,143 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 16:29:46,143 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 16:29:46,143 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 16:29:46,185 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 16:29:46,185 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 16:29:46,185 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 16:29:46,186 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 16:29:46,186 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 16:29:46,187 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 16:29:46,187 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:46,187 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A9FD6DD5CCEC4C298033735B61E6F580, all 1 inputs ready in: 00:00:00.000
2016-01-14 16:29:46,192 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788985824_0001_1_00_000000_0
2016-01-14 16:29:46,192 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A9FD6DD5CCEC4C298033735B61E6F580, mem on close (mb), free: 114, total: 461, max: 672
2016-01-14 16:29:46,192 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 16:29:46,192 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 16:29:46,192 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2016-01-14 16:29:46,192 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 16:29:46,197 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 16:29:46 Completed running task attempt: attempt_1452788985824_0001_1_00_000000_0
2016-01-14 16:29:46,202 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788985824_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:46,204 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788985824_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788985824_0001_1_00, eventCount=1
2016-01-14 16:29:46,205 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:46,205 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788985824_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A9FD6DD5CCEC4C298033735B61E6F580, taskAttemptId=attempt_1452788985824_0001_1_00_000000_0, startTime=1452788986092, finishTime=1452788986204, timeTaken=112, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788986187, Process_Duration=5, Process_End_Time=1452788986192, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 16:29:46,205 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788985824_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:46,205 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788985824_0001_1_00_000000_0
2016-01-14 16:29:46,205 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:46,206 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:46,209 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788985824_0001_1][Event:TASK_FINISHED]: vertexName=A9FD6DD5CCEC4C298033735B61E6F580, taskId=task_1452788985824_0001_1_00_000000, startTime=1452788986092, finishTime=1452788986206, timeTaken=114, status=SUCCEEDED, successfulAttemptID=attempt_1452788985824_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788986187, Process_Duration=5, Process_End_Time=1452788986192, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 16:29:46,209 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452788986187, Process_Duration=5, Process_End_Time=1452788986192, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 16:29:46,209 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788985824_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:46,210 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788985824_0001_1_00 [A9FD6DD5CCEC4C298033735B61E6F580] : 1
2016-01-14 16:29:46,210 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788985824_0001_1_00 [A9FD6DD5CCEC4C298033735B61E6F580]
2016-01-14 16:29:46,211 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:46,211 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:46,212 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788985824_0001_1][Event:VERTEX_FINISHED]: vertexName=A9FD6DD5CCEC4C298033735B61E6F580, vertexId=vertex_1452788985824_0001_1_00, initRequestedTime=1452788986034, initedTime=1452788986056, startRequestedTime=1452788986059, startedTime=1452788986059, finishTime=1452788986210, timeTaken=151, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788986187, Process_Duration=5, Process_End_Time=1452788986192, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452788986092, firstTasksToStart=[ task_1452788985824_0001_1_00_000000 ], lastTaskFinishTime=1452788986204, lastTasksToFinish=[ task_1452788985824_0001_1_00_000000 ], minTaskDuration=112, maxTaskDuration=112, avgTaskDuration=112.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788985824_0001_1_00_000000 ], longestDurationTasks=[ task_1452788985824_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:46,212 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788985824_0001_00_000001
2016-01-14 16:29:46,211 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:46,212 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788985824_0001_1_00 [A9FD6DD5CCEC4C298033735B61E6F580] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:46,214 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788985824_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:46,214 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788985824_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788985824_0001_00_000001, stoppedTime=1452788986214, exitStatus=0
2016-01-14 16:29:46,215 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788985824_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:46,215 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452788985824_0001_1_01 [C84E2E16ED6E44ABB4C6D3FC63AAD093] attempt: attempt_1452788985824_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 16:29:46,215 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:46,215 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: C84E2E16ED6E44ABB4C6D3FC63AAD093
2016-01-14 16:29:46,216 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788985824_0001_1_01 [C84E2E16ED6E44ABB4C6D3FC63AAD093]
2016-01-14 16:29:46,216 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788985824_0001_1_00 [A9FD6DD5CCEC4C298033735B61E6F580] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 16:29:46,216 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452788985824_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 16:29:46,216 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:46,217 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788985824_0001_1][Event:TASK_STARTED]: vertexName=C84E2E16ED6E44ABB4C6D3FC63AAD093, taskId=task_1452788985824_0001_1_01_000000, scheduledTime=1452788986216, launchTime=1452788986216
2016-01-14 16:29:46,217 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788985824_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:46,217 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788985824_0001_1_01 [C84E2E16ED6E44ABB4C6D3FC63AAD093]
2016-01-14 16:29:46,217 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788985824_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 16:29:46,218 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [0D68C3D7098245DFBFF00D5C923580BB/EDCAD390A7A346A69DAD890D3B3035C1] count/(1/1) ...splatform/testCount/count, VertexName: C84E2E16ED6E44ABB4C6D3FC63AAD093, VertexParallelism: 1, TaskAttemptID:attempt_1452788985824_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A9FD6DD5CCEC4C298033735B61E6F580, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=74CE8B0ABE1A40B3AF6875113FD0E97B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:46,218 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788985824_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:46,218 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:46,220 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788985824_0001_00_000002
2016-01-14 16:29:46,220 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788985824_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:46,223 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452788985824_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 16:29:46,223 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788985824_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 16:29:46,223 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452788985824_0001_00_000001, Exiting
2016-01-14 16:29:46,224 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788985824_0001_00_000001
2016-01-14 16:29:46,224 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788985824_0001_00_000001
2016-01-14 16:29:46,225 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788985824_0001_00_000002
2016-01-14 16:29:46,228 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 16:29:46,229 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788985824_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 16:29:46,229 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452788985824_0001_00_000001 completed successfully
2016-01-14 16:29:46,229 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788985824_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 16:29:46,230 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788985824_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:46,231 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:46,231 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788985824_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788985824_0001_00_000002, launchTime=1452788986230
2016-01-14 16:29:46,232 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788985824_0001_00_000002
2016-01-14 16:29:46,233 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788985824_0001_1_01_000000_0] to container: [container_1452788985824_0001_00_000002]
2016-01-14 16:29:46,233 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788985824_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:46,233 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788985824_0001_00_000002 given task: attempt_1452788985824_0001_1_01_000000_0
2016-01-14 16:29:46,234 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788985824_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788985824_0001_1_01_000000_0
2016-01-14 16:29:46,234 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788985824_0001_1_01_000000_0] started. Is using containerId: [container_1452788985824_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 16:29:46,234 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788985824_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C84E2E16ED6E44ABB4C6D3FC63AAD093, taskAttemptId=attempt_1452788985824_0001_1_01_000000_0, startTime=1452788986234, containerId=container_1452788985824_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788985824_0001_00_000002/teamcity, completedLogs=
2016-01-14 16:29:46,234 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788985824_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:46,235 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788985824_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:46,235 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:46 Starting to run new task attempt: attempt_1452788985824_0001_1_01_000000_0
2016-01-14 16:29:46,236 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788985824_0001_1_01_000000_0
2016-01-14 16:29:46,236 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:46,237 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:46,237 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:46,237 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:46,238 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:46,238 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:46,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [0D68C3D7098245DFBFF00D5C923580BB/EDCAD390A7A346A69DAD890D3B3035C1] count/(1/1) ...splatform/testCount/count, VertexName: C84E2E16ED6E44ABB4C6D3FC63AAD093, VertexParallelism: 1, TaskAttemptID:attempt_1452788985824_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A9FD6DD5CCEC4C298033735B61E6F580, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=74CE8B0ABE1A40B3AF6875113FD0E97B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:46,239 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:46,240 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788985824_0001_1_01_000000_0
2016-01-14 16:29:46,241 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:46,242 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:46,242 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788985824_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 16:29:46,243 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A9FD6DD5CCEC4C298033735B61E6F580, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 16:29:46,243 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:46,244 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=74CE8B0ABE1A40B3AF6875113FD0E97B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 16:29:46,245 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:46,245 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 74CE8B0ABE1A40B3AF6875113FD0E97B
2016-01-14 16:29:46,245 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 16:29:46,246 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 74CE8B0ABE1A40B3AF6875113FD0E97B
2016-01-14 16:29:46,250 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:46,253 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:46,253 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A9FD6DD5CCEC4C298033735B61E6F580
2016-01-14 16:29:46,265 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 16:29:46,265 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 16:29:46,265 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: A9FD6DD5CCEC4C298033735B61E6F580
2016-01-14 16:29:46,273 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A9FD6DD5CCEC4C298033735B61E6F580
2016-01-14 16:29:46,276 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 16:29:46,276 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 74CE8B0ABE1A40B3AF6875113FD0E97B
2016-01-14 16:29:46,278 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C84E2E16ED6E44ABB4C6D3FC63AAD093, ordinal: 1
2016-01-14 16:29:46,278 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C84E2E16ED6E44ABB4C6D3FC63AAD093, mem on start (mb), free: 103, total: 461, max: 672
2016-01-14 16:29:46,278 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:46,279 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:46,279 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:46,279 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:46,279 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:46,279 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 16:29:46,280 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:46,280 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 16:29:46,280 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 16:29:46,281 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 16:29:46,281 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 74CE8B0ABE1A40B3AF6875113FD0E97B, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 16:29:46,281 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A9FD6DD5CCEC4C298033735B61E6F580, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 16:29:46,282 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:46,282 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A9FD6DD5CCEC4C298033735B61E6F580
2016-01-14 16:29:46,283 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 16:29:46,283 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 16:29:46,284 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: A9FD6DD5CCEC4C298033735B61E6F580 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 16:29:46,284 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 16:29:46,284 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 16:29:46,285 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 16:29:46,286 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A9FD6DD5CCEC4C298033735B61E6F580
2016-01-14 16:29:46,286 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A9FD6DD5CCEC4C298033735B61E6F580 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:46,286 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 16:29:46,286 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:46,286 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:46,288 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788985824_0001_1_01_000000_0
2016-01-14 16:29:46,295 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-01-14 16:29:46,296 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: C771EE2E0FE043299DF05B81A0B20ABB
2016-01-14 16:29:46,296 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count"], id: 74CE8B0ABE1A40B3AF6875113FD0E97B
2016-01-14 16:29:46,292 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452788985824_0001_1_00_000000_0_10012, runDuration: 81294, ]
2016-01-14 16:29:46,297 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count"]
2016-01-14 16:29:46,298 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] C771EE2E0FE043299DF05B81A0B20ABB, for 1 inputs
2016-01-14 16:29:46,300 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452788985824_0001_1_00_000000_0_10012] to DISK_DIRECT, CompressedSize=463, DecompressedSize=459,EndTime=1452788986300, TimeTaken=2, Rate=0.22 MB/s
2016-01-14 16:29:46,300 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : A9FD6DD5CCEC4C298033735B61E6F580
2016-01-14 16:29:46,301 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 16:29:46,301 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [A9FD6DD5CCEC4C298033735B61E6F580] #3 in 3ms
2016-01-14 16:29:46,302 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [A9FD6DD5CCEC4C298033735B61E6F580] #3, status:false, isInterrupted:false
2016-01-14 16:29:46,303 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 16:29:46,304 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 463 bytes from disk
2016-01-14 16:29:46,304 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 16:29:46,305 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 16:29:46,305 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-01-14 16:29:46,306 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : A9FD6DD5CCEC4C298033735B61E6F580
2016-01-14 16:29:46,306 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 16:29:46,307 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C84E2E16ED6E44ABB4C6D3FC63AAD093, all 1 inputs ready in: 00:00:00.009
2016-01-14 16:29:46,308 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@55f88c89; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@30d2a277
2016-01-14 16:29:46,311 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 16:29:46,312 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 16:29:46,313 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452788985824_0001_1_01_000000_0
2016-01-14 16:29:46,314 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452788985824_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 16:29:46,315 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788985824_0001_1_01_000000_0
2016-01-14 16:29:46,315 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C84E2E16ED6E44ABB4C6D3FC63AAD093, mem on close (mb), free: 98, total: 461, max: 672
2016-01-14 16:29:46,315 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: A9FD6DD5CCEC4C298033735B61E6F580
2016-01-14 16:29:46,316 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 16:29:46,316 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 16:29:46,317 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 16:29:46,317 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 16:29:46,317 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 16:29:46 Completed running task attempt: attempt_1452788985824_0001_1_01_000000_0
2016-01-14 16:29:46,319 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788985824_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:46,321 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:46,324 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788985824_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C84E2E16ED6E44ABB4C6D3FC63AAD093, taskAttemptId=attempt_1452788985824_0001_1_01_000000_0, startTime=1452788986234, finishTime=1452788986320, timeTaken=86, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=22, FIRST_EVENT_RECEIVED=15, LAST_EVENT_RECEIVED=15, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788986307, Process_Duration=8, Process_End_Time=1452788986315, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-01-14 16:29:46,324 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788985824_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:46,324 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:46,325 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788985824_0001_1_01_000000_0
2016-01-14 16:29:46,325 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=22, FIRST_EVENT_RECEIVED=15, LAST_EVENT_RECEIVED=15][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452788986307, Process_Duration=8, Process_End_Time=1452788986315, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=17]]
2016-01-14 16:29:46,325 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:46,326 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:46,326 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:46,326 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788985824_0001_00_000002
2016-01-14 16:29:46,327 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788985824_0001_1][Event:TASK_FINISHED]: vertexName=C84E2E16ED6E44ABB4C6D3FC63AAD093, taskId=task_1452788985824_0001_1_01_000000, startTime=1452788986234, finishTime=1452788986325, timeTaken=91, status=SUCCEEDED, successfulAttemptID=attempt_1452788985824_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=22, FIRST_EVENT_RECEIVED=15, LAST_EVENT_RECEIVED=15, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788986307, Process_Duration=8, Process_End_Time=1452788986315, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-01-14 16:29:46,328 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788985824_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:46,328 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788985824_0001_1_01 [C84E2E16ED6E44ABB4C6D3FC63AAD093] : 1
2016-01-14 16:29:46,328 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788985824_0001_1_01 [C84E2E16ED6E44ABB4C6D3FC63AAD093]
2016-01-14 16:29:46,330 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:46,331 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788985824_0001_1][Event:VERTEX_FINISHED]: vertexName=C84E2E16ED6E44ABB4C6D3FC63AAD093, vertexId=vertex_1452788985824_0001_1_01, initRequestedTime=1452788986036, initedTime=1452788986052, startRequestedTime=1452788986070, startedTime=1452788986070, finishTime=1452788986328, timeTaken=258, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=22, FIRST_EVENT_RECEIVED=15, LAST_EVENT_RECEIVED=15, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788986307, Process_Duration=8, Process_End_Time=1452788986315, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1452788986234, firstTasksToStart=[ task_1452788985824_0001_1_01_000000 ], lastTaskFinishTime=1452788986320, lastTasksToFinish=[ task_1452788985824_0001_1_01_000000 ], minTaskDuration=86, maxTaskDuration=86, avgTaskDuration=86.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788985824_0001_1_01_000000 ], longestDurationTasks=[ task_1452788985824_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:46,332 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788985824_0001_1_01 [C84E2E16ED6E44ABB4C6D3FC63AAD093] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:46,332 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788985824_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:46,333 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788985824_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788985824_0001_00_000002, stoppedTime=1452788986332, exitStatus=0
2016-01-14 16:29:46,333 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788985824_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:46,333 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788985824_0001_1_01 [C84E2E16ED6E44ABB4C6D3FC63AAD093] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 16:29:46,333 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452788985824_0001_1
2016-01-14 16:29:46,334 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:46,334 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788985824_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452788985824_0001_1
2016-01-14 16:29:46,334 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452788985824_0001_1_00 [A9FD6DD5CCEC4C298033735B61E6F580]
2016-01-14 16:29:46,335 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 74CE8B0ABE1A40B3AF6875113FD0E97B for vertex: vertex_1452788985824_0001_1_01 [C84E2E16ED6E44ABB4C6D3FC63AAD093]
2016-01-14 16:29:46,337 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452788985824_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 16:29:46,338 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788985824_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 16:29:46,339 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452788985824_0001_00_000002, Exiting
2016-01-14 16:29:46,340 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788985824_0001_00_000002
2016-01-14 16:29:46,340 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788985824_0001_00_000002
2016-01-14 16:29:46,344 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788985824_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 16:29:46,344 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452788985824_0001_00_000002 completed successfully
2016-01-14 16:29:46,352 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452788985824_0001_1, queueSize=0
2016-01-14 16:29:46,352 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 16:29:46,354 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788985824_0001_1][Event:DAG_FINISHED]: dagId=dag_1452788985824_0001_1, startTime=1452788986033, finishTime=1452788986349, timeTaken=316, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=940, FILE_BYTES_WRITTEN=1297, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=966787072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=22, FIRST_EVENT_RECEIVED=15, LAST_EVENT_RECEIVED=15, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905577972494, Process_Duration=13, Process_End_Time=2905577972507, Read_Duration=4, Tuples_Read=40, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-01-14 16:29:46,354 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452788985824_0001_1 finished with state: SUCCEEDED
2016-01-14 16:29:46,354 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788985824_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 16:29:46,355 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788985824_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 16:29:46,356 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452788985824_0001_00_000002
2016-01-14 16:29:46 Completed Dag: dag_1452788985824_0001_1
2016-01-14 16:29:46,357 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452788985824_0001_1, dagState=SUCCEEDED
2016-01-14 16:29:46,357 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788985824_0001_1_post
2016-01-14 16:29:46,357 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:46,357 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 16:29:46,364 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...splatform/testCount/count, applicationId=application_1452788985824_0001
2016-01-14 16:29:46,367 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 16:29:46,367 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 16:29:46,367 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 16:29:46,367 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 16:29:46,368 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 16:29:46,368 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 16:29:46,369 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 16:29:46,369 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 16:29:46,369 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 16:29:46,369 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 16:29:46,370 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 16:29:46,371 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 16:29:46,371 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 16:29:46,377 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0D68C3D7098245DFBFF00D5C923580BB/EDCAD390A7A346A69DAD890D3B3035C1/.tez/application_1452788985824_0001
2016-01-14 16:29:46,378 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] completed in: 00:00.584
2016-01-14 16:29:46,475 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount/count
2016-01-14 16:29:46,517 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 16:29:46,518 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 16:29:46,582 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-01-14 16:29:46,595 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.033
2016-01-14 16:29:46,596 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 16:29:46,596 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 16:29:46,596 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 16:29:46,626 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-01-14 16:29:46,627 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap
2016-01-14 16:29:46,645 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 16:29:46,645 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 16:29:46,645 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap"]
2016-01-14 16:29:46,646 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 16:29:46,646 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 16:29:46,646 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 16:29:46,647 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...pesplatform/testSwap/swap
2016-01-14 16:29:46,647 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 16:29:46,667 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 16:29:46,672 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:46,673 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 16:29:46,673 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 16:29:46,683 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B3FC1E8E204D4FEBAAE4BBD446ECC0F7/A9DDE8BE3D6045548A5C79CC878C23DA/.tez/application_1452788986673_0001 doesn't exist and is created
2016-01-14 16:29:46,705 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 16:29:46,705 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 16:29:46,707 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B3FC1E8E204D4FEBAAE4BBD446ECC0F7/A9DDE8BE3D6045548A5C79CC878C23DA/.tez/application_1452788986673_0001
2016-01-14 16:29:46,714 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452788986673_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:46,731 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 16:29:46,732 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 16:29:46,735 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 16:29:46,735 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 16:29:46,735 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B3FC1E8E204D4FEBAAE4BBD446ECC0F7/A9DDE8BE3D6045548A5C79CC878C23DA/.tez/application_1452788986673_0001/localmode-log-dir
2016-01-14 16:29:46,736 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B3FC1E8E204D4FEBAAE4BBD446ECC0F7/A9DDE8BE3D6045548A5C79CC878C23DA/.tez/application_1452788986673_0001/localmode-log-dir/history.txt.appattempt_1452788986673_0001_000000, maxErrors=10
2016-01-14 16:29:46,736 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 16:29:46,736 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452788986673_0001_000000, appSubmitTime=1452788986713, launchTime=1452788986713
2016-01-14 16:29:46,763 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 16:29:46,766 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 16:29:46,767 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:57376
2016-01-14 16:29:46,768 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 16:29:46,769 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 16:29:46,773 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 16:29:46,777 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452788986673_0001_000000, startTime=1452788986777
2016-01-14 16:29:46,778 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 16:29:46,778 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 16:29:46,806 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 16:29:46,806 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 16:29:46,806 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...pesplatform/testSwap/swap, applicationId=application_1452788986673_0001, dagName=[B3FC1E8E204D4FEBAAE4BBD446ECC0F7/A9DDE8BE3D6045548A5C79CC878C23DA] test/(1/1) ...pesplatform/testSwap/swap
2016-01-14 16:29:46,826 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [B3FC1E8E204D4FEBAAE4BBD446ECC0F7/A9DDE8BE3D6045548A5C79CC878C23DA] test/(1/1) ...pesplatform/testSwap/swap
2016-01-14 16:29:46,829 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452788986673_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B3FC1E8E204D4FEBAAE4BBD446ECC0F7/A9DDE8BE3D6045548A5C79CC878C23DA/.tez/application_1452788986673_0001/localmode-log-dir/dag_1452788986673_0001_1.dot
2016-01-14 16:29:46,830 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B3FC1E8E204D4FEBAAE4BBD446ECC0F7/A9DDE8BE3D6045548A5C79CC878C23DA/.tez/application_1452788986673_0001/localmode-log-dir/dag_1452788986673_0001_1-tez-dag.pb.txt
2016-01-14 16:29:46,858 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788986673_0001_1
2016-01-14 16:29:46,858 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:46,858 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [B3FC1E8E204D4FEBAAE4BBD446ECC0F7/A9DDE8BE3D6045548A5C79CC878C23DA] test/(1/1) ...pesplatform/testSwap/swap
2016-01-14 16:29:46 Running Dag: dag_1452788986673_0001_1
2016-01-14 16:29:46,875 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788986673_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452788986673_0001_1, submitTime=1452788986827
2016-01-14 16:29:46,875 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 16:29:46,875 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B2EF740D4CDB4152A0F24EB7B0F6FB4D
2016-01-14 16:29:46,876 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 16:29:46,877 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788986673_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452788986673_0001_1, initTime=1452788986875
2016-01-14 16:29:46,877 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 16:29:46,878 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788986673_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 16:29:46,879 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788986673_0001_1][Event:DAG_STARTED]: dagID=dag_1452788986673_0001_1, startTime=1452788986879
2016-01-14 16:29:46,879 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 16:29:46,879 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 16:29:46,880 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...pesplatform/testSwap/swap, applicationId=application_1452788986673_0001, dagName=[B3FC1E8E204D4FEBAAE4BBD446ECC0F7/A9DDE8BE3D6045548A5C79CC878C23DA] test/(1/1) ...pesplatform/testSwap/swap
2016-01-14 16:29:46,880 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788986673_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 16:29:46,880 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B2EF740D4CDB4152A0F24EB7B0F6FB4D : {A5E1E317AB1542FAB80E9417216D4015={InputName=A5E1E317AB1542FAB80E9417216D4015}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 16:29:46,880 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: A5E1E317AB1542FAB80E9417216D4015, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 16:29:46,880 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452788986673_0001_1_00 [B2EF740D4CDB4152A0F24EB7B0F6FB4D]
2016-01-14 16:29:46,880 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452788986673_0001_1_00 [B2EF740D4CDB4152A0F24EB7B0F6FB4D]
2016-01-14 16:29:46,880 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452788986673_0001_1_00 [B2EF740D4CDB4152A0F24EB7B0F6FB4D]
2016-01-14 16:29:46,881 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452788986673_0001_1_00 [B2EF740D4CDB4152A0F24EB7B0F6FB4D]. Starting root input initializers: 1
2016-01-14 16:29:46,881 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788986673_0001_1_00 [B2EF740D4CDB4152A0F24EB7B0F6FB4D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 16:29:46,881 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: A5E1E317AB1542FAB80E9417216D4015 on vertex vertex_1452788986673_0001_1_00 [B2EF740D4CDB4152A0F24EB7B0F6FB4D]
2016-01-14 16:29:46,884 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 16:29:46,884 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input A5E1E317AB1542FAB80E9417216D4015 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 16:29:46,887 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 16:29:46,887 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 16:29:46,890 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-01-14 16:29:46,890 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: A5E1E317AB1542FAB80E9417216D4015 on vertex vertex_1452788986673_0001_1_00 [B2EF740D4CDB4152A0F24EB7B0F6FB4D]
2016-01-14 16:29:46,890 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788986673_0001_1_00 [B2EF740D4CDB4152A0F24EB7B0F6FB4D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:46,890 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {A5E1E317AB1542FAB80E9417216D4015=forAllWorkUnits=true, update=[1]}
2016-01-14 16:29:46,891 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452788986673_0001_1_00 [B2EF740D4CDB4152A0F24EB7B0F6FB4D] parallelism set to 1
2016-01-14 16:29:46,891 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788986673_0001_1_00 [B2EF740D4CDB4152A0F24EB7B0F6FB4D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:46,891 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788986673_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788986673_0001_1_00, eventCount=1
2016-01-14 16:29:46,891 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452788986673_0001_1_00 [B2EF740D4CDB4152A0F24EB7B0F6FB4D]
2016-01-14 16:29:46,892 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=0C3C4B9BB6D94EE688A2656A72D78EF8, vertexId=vertex_1452788986673_0001_1_00 [B2EF740D4CDB4152A0F24EB7B0F6FB4D], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 16:29:46,892 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=0C3C4B9BB6D94EE688A2656A72D78EF8, vertexId=vertex_1452788986673_0001_1_00 [B2EF740D4CDB4152A0F24EB7B0F6FB4D]
2016-01-14 16:29:46,894 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=0C3C4B9BB6D94EE688A2656A72D78EF8, vertexName=B2EF740D4CDB4152A0F24EB7B0F6FB4D, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 16:29:46,895 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=0C3C4B9BB6D94EE688A2656A72D78EF8, vertexId=vertex_1452788986673_0001_1_00 [B2EF740D4CDB4152A0F24EB7B0F6FB4D]
2016-01-14 16:29:46,897 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788986673_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B2EF740D4CDB4152A0F24EB7B0F6FB4D, vertexId=vertex_1452788986673_0001_1_00, initRequestedTime=1452788986880, initedTime=1452788986897, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 16:29:46,897 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788986673_0001_1_00 [B2EF740D4CDB4152A0F24EB7B0F6FB4D] with distanceFromRoot: 0
2016-01-14 16:29:46,897 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:46,897 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788986673_0001_1_00 [B2EF740D4CDB4152A0F24EB7B0F6FB4D] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 16:29:46,898 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B2EF740D4CDB4152A0F24EB7B0F6FB4D
2016-01-14 16:29:46,898 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788986673_0001_1_00 [B2EF740D4CDB4152A0F24EB7B0F6FB4D]
2016-01-14 16:29:46,898 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788986673_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788986673_0001_1_00, startRequestedTime=1452788986898, startedTime=1452788986898
2016-01-14 16:29:46,899 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788986673_0001_1_00 [B2EF740D4CDB4152A0F24EB7B0F6FB4D] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:46,898 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:46,900 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:46,900 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788986673_0001_1][Event:TASK_STARTED]: vertexName=B2EF740D4CDB4152A0F24EB7B0F6FB4D, taskId=task_1452788986673_0001_1_00_000000, scheduledTime=1452788986900, launchTime=1452788986900
2016-01-14 16:29:46,900 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788986673_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:46,901 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788986673_0001_1_00 [B2EF740D4CDB4152A0F24EB7B0F6FB4D]
2016-01-14 16:29:46,901 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [B3FC1E8E204D4FEBAAE4BBD446ECC0F7/A9DDE8BE3D6045548A5C79CC878C23DA] test/(1/1) ...pesplatform/testSwap/swap, VertexName: B2EF740D4CDB4152A0F24EB7B0F6FB4D, VertexParallelism: 1, TaskAttemptID:attempt_1452788986673_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A5E1E317AB1542FAB80E9417216D4015, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=0C3C4B9BB6D94EE688A2656A72D78EF8, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:46,902 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788986673_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:46,902 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:46,903 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 16:29:46,904 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 16:29:46,904 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 16:29:46,905 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788986673_0001_00_000001
2016-01-14 16:29:46,905 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788986673_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:46,906 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788986673_0001_00_000001
2016-01-14 16:29:46,906 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 16:29:46,907 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788986673_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:46,907 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:46,908 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788986673_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788986673_0001_00_000001, launchTime=1452788986907
2016-01-14 16:29:46,908 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788986673_0001_00_000001
2016-01-14 16:29:46,910 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452788986673_0001, with dag id: dag_1452788986673_0001_1
2016-01-14 16:29:46,912 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788986673_0001_1_00_000000_0] to container: [container_1452788986673_0001_00_000001]
2016-01-14 16:29:46,912 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788986673_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:46,912 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788986673_0001_00_000001 given task: attempt_1452788986673_0001_1_00_000000_0
2016-01-14 16:29:46,915 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788986673_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788986673_0001_1_00_000000_0
2016-01-14 16:29:46,915 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788986673_0001_1_00_000000_0] started. Is using containerId: [container_1452788986673_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 16:29:46,915 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788986673_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B2EF740D4CDB4152A0F24EB7B0F6FB4D, taskAttemptId=attempt_1452788986673_0001_1_00_000000_0, startTime=1452788986915, containerId=container_1452788986673_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788986673_0001_00_000001/teamcity, completedLogs=
2016-01-14 16:29:46,915 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788986673_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:46 Starting to run new task attempt: attempt_1452788986673_0001_1_00_000000_0
2016-01-14 16:29:46,915 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788986673_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:46,915 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788986673_0001_1_00_000000_0
2016-01-14 16:29:46,915 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:46,917 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:46,917 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:46,917 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:46,917 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:46,917 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:46,918 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:46,918 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B3FC1E8E204D4FEBAAE4BBD446ECC0F7/A9DDE8BE3D6045548A5C79CC878C23DA] test/(1/1) ...pesplatform/testSwap/swap, VertexName: B2EF740D4CDB4152A0F24EB7B0F6FB4D, VertexParallelism: 1, TaskAttemptID:attempt_1452788986673_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A5E1E317AB1542FAB80E9417216D4015, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=0C3C4B9BB6D94EE688A2656A72D78EF8, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:46,918 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:46,919 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788986673_0001_1_00_000000_0
2016-01-14 16:29:46,920 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:46,920 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788986673_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 16:29:46,920 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:46,922 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A5E1E317AB1542FAB80E9417216D4015, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 16:29:46,923 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:46,922 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=0C3C4B9BB6D94EE688A2656A72D78EF8, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 16:29:46,923 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:46,924 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 0C3C4B9BB6D94EE688A2656A72D78EF8
2016-01-14 16:29:46,924 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 16:29:46,924 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 0C3C4B9BB6D94EE688A2656A72D78EF8
2016-01-14 16:29:46,922 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:46,923 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A5E1E317AB1542FAB80E9417216D4015
2016-01-14 16:29:46,925 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: A5E1E317AB1542FAB80E9417216D4015
2016-01-14 16:29:46,928 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 16:29:46,928 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: A5E1E317AB1542FAB80E9417216D4015
2016-01-14 16:29:46,928 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A5E1E317AB1542FAB80E9417216D4015
2016-01-14 16:29:46,930 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:46,937 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 16:29:46,937 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 0C3C4B9BB6D94EE688A2656A72D78EF8
2016-01-14 16:29:46,940 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B2EF740D4CDB4152A0F24EB7B0F6FB4D, ordinal: 0
2016-01-14 16:29:46,940 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B2EF740D4CDB4152A0F24EB7B0F6FB4D, mem on start (mb), free: 240, total: 461, max: 672
2016-01-14 16:29:46,940 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:46,940 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:46,940 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:46,940 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:46,940 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:46,941 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 16:29:46,941 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:46,941 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 16:29:46,941 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:46,941 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 16:29:46,942 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-01-14 16:29:46,942 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 0C3C4B9BB6D94EE688A2656A72D78EF8, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 16:29:46,942 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A5E1E317AB1542FAB80E9417216D4015, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 16:29:46,942 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:46,942 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A5E1E317AB1542FAB80E9417216D4015 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:46,942 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A5E1E317AB1542FAB80E9417216D4015
2016-01-14 16:29:46,942 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:46,943 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A5E1E317AB1542FAB80E9417216D4015
2016-01-14 16:29:46,942 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 16:29:46,943 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:46,943 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:46,943 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788986673_0001_1_00_000000_0
2016-01-14 16:29:46,945 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 16:29:46,946 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 16:29:46,946 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@68e0f4b6
2016-01-14 16:29:46,946 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: A5E1E317AB1542FAB80E9417216D4015
2016-01-14 16:29:46,947 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap"], id: 0C3C4B9BB6D94EE688A2656A72D78EF8
2016-01-14 16:29:46,948 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap"]
2016-01-14 16:29:46,948 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 16:29:46,949 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:46,948 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@68e0f4b6
2016-01-14 16:29:46,949 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 16:29:46,950 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 16:29:46,950 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B2EF740D4CDB4152A0F24EB7B0F6FB4D, all 1 inputs ready in: 00:00:00.000
2016-01-14 16:29:46,953 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 16:29:46,954 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 16:29:46,954 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452788986673_0001_1_00_000000_0
2016-01-14 16:29:46,955 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452788986673_0001_1_00_000000_0 given a go for committing the task output.
2016-01-14 16:29:46,956 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788986673_0001_1_00_000000_0
2016-01-14 16:29:46,956 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B2EF740D4CDB4152A0F24EB7B0F6FB4D, mem on close (mb), free: 238, total: 461, max: 672
2016-01-14 16:29:46 Completed running task attempt: attempt_1452788986673_0001_1_00_000000_0
2016-01-14 16:29:46,956 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788986673_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:46,958 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10][cascading.flow.SliceCounters Process_Begin_Time=1452788986950, Process_Duration=5, Process_End_Time=1452788986955, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10, Tuples_Written=10]]
2016-01-14 16:29:46,959 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:46,959 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:46,959 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788986673_0001_00_000001
2016-01-14 16:29:46,958 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788986673_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B2EF740D4CDB4152A0F24EB7B0F6FB4D, taskAttemptId=attempt_1452788986673_0001_1_00_000000_0, startTime=1452788986915, finishTime=1452788986957, timeTaken=42, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452788986950, Process_Duration=5, Process_End_Time=1452788986955, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-01-14 16:29:46,960 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788986673_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:46,958 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:46,961 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:46,961 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788986673_0001_1_00_000000_0
2016-01-14 16:29:46,962 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:46,962 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788986673_0001_1][Event:TASK_FINISHED]: vertexName=B2EF740D4CDB4152A0F24EB7B0F6FB4D, taskId=task_1452788986673_0001_1_00_000000, startTime=1452788986915, finishTime=1452788986961, timeTaken=46, status=SUCCEEDED, successfulAttemptID=attempt_1452788986673_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452788986950, Process_Duration=5, Process_End_Time=1452788986955, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-01-14 16:29:46,962 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788986673_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:46,962 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788986673_0001_1_00 [B2EF740D4CDB4152A0F24EB7B0F6FB4D] : 1
2016-01-14 16:29:46,962 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788986673_0001_1_00 [B2EF740D4CDB4152A0F24EB7B0F6FB4D]
2016-01-14 16:29:46,963 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:46,964 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788986673_0001_1][Event:VERTEX_FINISHED]: vertexName=B2EF740D4CDB4152A0F24EB7B0F6FB4D, vertexId=vertex_1452788986673_0001_1_00, initRequestedTime=1452788986880, initedTime=1452788986897, startRequestedTime=1452788986898, startedTime=1452788986898, finishTime=1452788986963, timeTaken=65, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452788986950, Process_Duration=5, Process_End_Time=1452788986955, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10, vertexStats=firstTaskStartTime=1452788986915, firstTasksToStart=[ task_1452788986673_0001_1_00_000000 ], lastTaskFinishTime=1452788986957, lastTasksToFinish=[ task_1452788986673_0001_1_00_000000 ], minTaskDuration=42, maxTaskDuration=42, avgTaskDuration=42.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788986673_0001_1_00_000000 ], longestDurationTasks=[ task_1452788986673_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:46,964 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788986673_0001_1_00 [B2EF740D4CDB4152A0F24EB7B0F6FB4D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:46,964 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788986673_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:46,965 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788986673_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788986673_0001_00_000001, stoppedTime=1452788986965, exitStatus=0
2016-01-14 16:29:46,965 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:46,965 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788986673_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:46,965 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788986673_0001_1_00 [B2EF740D4CDB4152A0F24EB7B0F6FB4D] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-01-14 16:29:46,966 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452788986673_0001_1
2016-01-14 16:29:46,966 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788986673_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452788986673_0001_1
2016-01-14 16:29:46,966 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 0C3C4B9BB6D94EE688A2656A72D78EF8 for vertex: vertex_1452788986673_0001_1_00 [B2EF740D4CDB4152A0F24EB7B0F6FB4D]
2016-01-14 16:29:46,972 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452788986673_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 16:29:46,973 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788986673_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 16:29:46,973 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452788986673_0001_00_000001, Exiting
2016-01-14 16:29:46,973 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788986673_0001_00_000001
2016-01-14 16:29:46,973 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788986673_0001_00_000001
2016-01-14 16:29:46,976 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452788986673_0001_1, queueSize=0
2016-01-14 16:29:46,977 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 16:29:46,977 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788986673_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 16:29:46,977 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788986673_0001_1][Event:DAG_FINISHED]: dagId=dag_1452788986673_0001_1, startTime=1452788986879, finishTime=1452788986974, timeTaken=95, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452788986950, Process_Duration=5, Process_End_Time=1452788986955, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-01-14 16:29:46,977 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452788986673_0001_00_000001 completed successfully
2016-01-14 16:29:46,978 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452788986673_0001_1 finished with state: SUCCEEDED
2016-01-14 16:29:46,978 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788986673_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 16:29:46 Completed Dag: dag_1452788986673_0001_1
2016-01-14 16:29:46,979 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452788986673_0001_1, dagState=SUCCEEDED
2016-01-14 16:29:46,979 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788986673_0001_1_post
2016-01-14 16:29:46,979 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:46,979 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 16:29:46,979 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452788986673_0001_00_000001
2016-01-14 16:29:46,980 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788986673_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 16:29:46,986 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...pesplatform/testSwap/swap, applicationId=application_1452788986673_0001
2016-01-14 16:29:46,988 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 16:29:46,988 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 16:29:46,988 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 16:29:46,989 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 16:29:46,989 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 16:29:46,990 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 16:29:46,990 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 16:29:46,991 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 16:29:46,991 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 16:29:46,991 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 16:29:46,992 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 16:29:46,992 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 16:29:46,993 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 16:29:46,999 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B3FC1E8E204D4FEBAAE4BBD446ECC0F7/A9DDE8BE3D6045548A5C79CC878C23DA/.tez/application_1452788986673_0001
2016-01-14 16:29:47,000 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:00.355
2016-01-14 16:29:47,041 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSwap/swap
2016-01-14 16:29:47,070 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 16:29:47,071 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 16:29:47,120 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-01-14 16:29:47,127 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.025
2016-01-14 16:29:47,127 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 16:29:47,127 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 16:29:47,129 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 16:29:47,147 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower+upper.txt
2016-01-14 16:29:47,148 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped
2016-01-14 16:29:47,164 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 16:29:47,164 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower+upper.txt"]
2016-01-14 16:29:47,164 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped"]
2016-01-14 16:29:47,164 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 16:29:47,164 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 16:29:47,165 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 16:29:47,165 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...orm/testUnGroup/ungrouped
2016-01-14 16:29:47,166 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 16:29:47,180 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 16:29:47,185 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:47,185 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 16:29:47,185 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 16:29:47,195 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/FC740EBBE19E42FAA54FC3C93434CADE/AC645A05C347467FB0134583CE34BDB5/.tez/application_1452788987185_0001 doesn't exist and is created
2016-01-14 16:29:47,214 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 16:29:47,215 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 16:29:47,216 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/FC740EBBE19E42FAA54FC3C93434CADE/AC645A05C347467FB0134583CE34BDB5/.tez/application_1452788987185_0001
2016-01-14 16:29:47,220 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452788987185_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:47,232 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 16:29:47,233 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 16:29:47,236 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 16:29:47,236 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 16:29:47,236 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/FC740EBBE19E42FAA54FC3C93434CADE/AC645A05C347467FB0134583CE34BDB5/.tez/application_1452788987185_0001/localmode-log-dir
2016-01-14 16:29:47,236 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/FC740EBBE19E42FAA54FC3C93434CADE/AC645A05C347467FB0134583CE34BDB5/.tez/application_1452788987185_0001/localmode-log-dir/history.txt.appattempt_1452788987185_0001_000000, maxErrors=10
2016-01-14 16:29:47,236 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 16:29:47,238 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452788987185_0001_000000, appSubmitTime=1452788987219, launchTime=1452788987220
2016-01-14 16:29:47,248 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 16:29:47,252 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 16:29:47,254 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:60912
2016-01-14 16:29:47,256 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 16:29:47,256 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 16:29:47,257 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 16:29:47,259 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452788987185_0001_000000, startTime=1452788987259
2016-01-14 16:29:47,259 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 16:29:47,260 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 16:29:47,315 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 16:29:47,315 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 16:29:47,315 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...orm/testUnGroup/ungrouped, applicationId=application_1452788987185_0001, dagName=[FC740EBBE19E42FAA54FC3C93434CADE/AC645A05C347467FB0134583CE34BDB5] test/(1/1) ...orm/testUnGroup/ungrouped
2016-01-14 16:29:47,335 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [FC740EBBE19E42FAA54FC3C93434CADE/AC645A05C347467FB0134583CE34BDB5] test/(1/1) ...orm/testUnGroup/ungrouped
2016-01-14 16:29:47,337 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452788987185_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/FC740EBBE19E42FAA54FC3C93434CADE/AC645A05C347467FB0134583CE34BDB5/.tez/application_1452788987185_0001/localmode-log-dir/dag_1452788987185_0001_1.dot
2016-01-14 16:29:47,338 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/FC740EBBE19E42FAA54FC3C93434CADE/AC645A05C347467FB0134583CE34BDB5/.tez/application_1452788987185_0001/localmode-log-dir/dag_1452788987185_0001_1-tez-dag.pb.txt
2016-01-14 16:29:47,361 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788987185_0001_1
2016-01-14 16:29:47,362 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:47,362 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [FC740EBBE19E42FAA54FC3C93434CADE/AC645A05C347467FB0134583CE34BDB5] test/(1/1) ...orm/testUnGroup/ungrouped
2016-01-14 16:29:47 Running Dag: dag_1452788987185_0001_1
2016-01-14 16:29:47,375 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 16:29:47,376 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987185_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452788987185_0001_1, submitTime=1452788987335
2016-01-14 16:29:47,376 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BCA6E44ACBC4488590B8A3AA73927FE5
2016-01-14 16:29:47,376 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 16:29:47,377 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 16:29:47,377 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987185_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452788987185_0001_1, initTime=1452788987376
2016-01-14 16:29:47,377 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788987185_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 16:29:47,378 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987185_0001_1][Event:DAG_STARTED]: dagID=dag_1452788987185_0001_1, startTime=1452788987378
2016-01-14 16:29:47,378 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 16:29:47,378 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788987185_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 16:29:47,378 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BCA6E44ACBC4488590B8A3AA73927FE5 : {A580F15BA58443DE826FAB4BD2B077A9={InputName=A580F15BA58443DE826FAB4BD2B077A9}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 16:29:47,379 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: A580F15BA58443DE826FAB4BD2B077A9, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 16:29:47,379 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452788987185_0001_1_00 [BCA6E44ACBC4488590B8A3AA73927FE5]
2016-01-14 16:29:47,379 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452788987185_0001_1_00 [BCA6E44ACBC4488590B8A3AA73927FE5]
2016-01-14 16:29:47,379 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 16:29:47,379 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452788987185_0001_1_00 [BCA6E44ACBC4488590B8A3AA73927FE5]
2016-01-14 16:29:47,379 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452788987185_0001_1_00 [BCA6E44ACBC4488590B8A3AA73927FE5]. Starting root input initializers: 1
2016-01-14 16:29:47,381 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788987185_0001_1_00 [BCA6E44ACBC4488590B8A3AA73927FE5] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 16:29:47,382 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: A580F15BA58443DE826FAB4BD2B077A9 on vertex vertex_1452788987185_0001_1_00 [BCA6E44ACBC4488590B8A3AA73927FE5]
2016-01-14 16:29:47,382 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...orm/testUnGroup/ungrouped, applicationId=application_1452788987185_0001, dagName=[FC740EBBE19E42FAA54FC3C93434CADE/AC645A05C347467FB0134583CE34BDB5] test/(1/1) ...orm/testUnGroup/ungrouped
2016-01-14 16:29:47,384 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 16:29:47,384 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input A580F15BA58443DE826FAB4BD2B077A9 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 16:29:47,385 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 16:29:47,385 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 16:29:47,387 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 412
2016-01-14 16:29:47,388 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: A580F15BA58443DE826FAB4BD2B077A9 on vertex vertex_1452788987185_0001_1_00 [BCA6E44ACBC4488590B8A3AA73927FE5]
2016-01-14 16:29:47,388 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788987185_0001_1_00 [BCA6E44ACBC4488590B8A3AA73927FE5] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:47,388 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {A580F15BA58443DE826FAB4BD2B077A9=forAllWorkUnits=true, update=[1]}
2016-01-14 16:29:47,388 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452788987185_0001_1_00 [BCA6E44ACBC4488590B8A3AA73927FE5] parallelism set to 1
2016-01-14 16:29:47,388 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788987185_0001_1_00 [BCA6E44ACBC4488590B8A3AA73927FE5] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:47,389 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987185_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788987185_0001_1_00, eventCount=1
2016-01-14 16:29:47,389 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452788987185_0001_1_00 [BCA6E44ACBC4488590B8A3AA73927FE5]
2016-01-14 16:29:47,390 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=9D68F129978F402299C369E042841A38, vertexId=vertex_1452788987185_0001_1_00 [BCA6E44ACBC4488590B8A3AA73927FE5], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 16:29:47,390 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=9D68F129978F402299C369E042841A38, vertexId=vertex_1452788987185_0001_1_00 [BCA6E44ACBC4488590B8A3AA73927FE5]
2016-01-14 16:29:47,391 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=9D68F129978F402299C369E042841A38, vertexName=BCA6E44ACBC4488590B8A3AA73927FE5, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 16:29:47,392 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=9D68F129978F402299C369E042841A38, vertexId=vertex_1452788987185_0001_1_00 [BCA6E44ACBC4488590B8A3AA73927FE5]
2016-01-14 16:29:47,394 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987185_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BCA6E44ACBC4488590B8A3AA73927FE5, vertexId=vertex_1452788987185_0001_1_00, initRequestedTime=1452788987378, initedTime=1452788987393, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 16:29:47,394 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788987185_0001_1_00 [BCA6E44ACBC4488590B8A3AA73927FE5] with distanceFromRoot: 0
2016-01-14 16:29:47,394 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:47,394 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788987185_0001_1_00 [BCA6E44ACBC4488590B8A3AA73927FE5] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 16:29:47,395 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BCA6E44ACBC4488590B8A3AA73927FE5
2016-01-14 16:29:47,395 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788987185_0001_1_00 [BCA6E44ACBC4488590B8A3AA73927FE5]
2016-01-14 16:29:47,395 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:47,395 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987185_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788987185_0001_1_00, startRequestedTime=1452788987395, startedTime=1452788987395
2016-01-14 16:29:47,396 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788987185_0001_1_00 [BCA6E44ACBC4488590B8A3AA73927FE5] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:47,396 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:47,397 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987185_0001_1][Event:TASK_STARTED]: vertexName=BCA6E44ACBC4488590B8A3AA73927FE5, taskId=task_1452788987185_0001_1_00_000000, scheduledTime=1452788987396, launchTime=1452788987396
2016-01-14 16:29:47,397 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788987185_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:47,397 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788987185_0001_1_00 [BCA6E44ACBC4488590B8A3AA73927FE5]
2016-01-14 16:29:47,398 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [FC740EBBE19E42FAA54FC3C93434CADE/AC645A05C347467FB0134583CE34BDB5] test/(1/1) ...orm/testUnGroup/ungrouped, VertexName: BCA6E44ACBC4488590B8A3AA73927FE5, VertexParallelism: 1, TaskAttemptID:attempt_1452788987185_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A580F15BA58443DE826FAB4BD2B077A9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=9D68F129978F402299C369E042841A38, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:47,398 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788987185_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:47,399 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:47,399 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 16:29:47,400 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 16:29:47,400 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 16:29:47,401 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788987185_0001_00_000001
2016-01-14 16:29:47,401 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788987185_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:47,403 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788987185_0001_00_000001
2016-01-14 16:29:47,403 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 16:29:47,404 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788987185_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:47,404 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:47,405 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987185_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788987185_0001_00_000001, launchTime=1452788987404
2016-01-14 16:29:47,406 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788987185_0001_00_000001
2016-01-14 16:29:47,407 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788987185_0001_1_00_000000_0] to container: [container_1452788987185_0001_00_000001]
2016-01-14 16:29:47,407 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788987185_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:47,408 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788987185_0001_1_00_000000_0] started. Is using containerId: [container_1452788987185_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 16:29:47,408 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987185_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BCA6E44ACBC4488590B8A3AA73927FE5, taskAttemptId=attempt_1452788987185_0001_1_00_000000_0, startTime=1452788987408, containerId=container_1452788987185_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788987185_0001_00_000001/teamcity, completedLogs=
2016-01-14 16:29:47,408 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:47,408 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788987185_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:47,408 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788987185_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:47,409 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788987185_0001_00_000001 given task: attempt_1452788987185_0001_1_00_000000_0
2016-01-14 16:29:47,409 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788987185_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788987185_0001_1_00_000000_0
2016-01-14 16:29:47 Starting to run new task attempt: attempt_1452788987185_0001_1_00_000000_0
2016-01-14 16:29:47,410 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788987185_0001_1_00_000000_0
2016-01-14 16:29:47,410 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:47,410 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:47,411 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:47,411 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:47,411 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:47,411 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:47,411 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [FC740EBBE19E42FAA54FC3C93434CADE/AC645A05C347467FB0134583CE34BDB5] test/(1/1) ...orm/testUnGroup/ungrouped, VertexName: BCA6E44ACBC4488590B8A3AA73927FE5, VertexParallelism: 1, TaskAttemptID:attempt_1452788987185_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A580F15BA58443DE826FAB4BD2B077A9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=9D68F129978F402299C369E042841A38, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:47,411 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:47,413 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788987185_0001_1_00_000000_0
2016-01-14 16:29:47,414 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:47,414 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:47,415 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788987185_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 16:29:47,415 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A580F15BA58443DE826FAB4BD2B077A9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 16:29:47,416 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:47,416 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:47,416 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A580F15BA58443DE826FAB4BD2B077A9
2016-01-14 16:29:47,417 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: A580F15BA58443DE826FAB4BD2B077A9
2016-01-14 16:29:47,417 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452788987185_0001, with dag id: dag_1452788987185_0001_1
2016-01-14 16:29:47,422 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:47,423 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 16:29:47,424 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: A580F15BA58443DE826FAB4BD2B077A9
2016-01-14 16:29:47,424 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A580F15BA58443DE826FAB4BD2B077A9
2016-01-14 16:29:47,424 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=9D68F129978F402299C369E042841A38, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 16:29:47,426 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:47,426 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 9D68F129978F402299C369E042841A38
2016-01-14 16:29:47,427 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 16:29:47,427 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 9D68F129978F402299C369E042841A38
2016-01-14 16:29:47,433 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BCA6E44ACBC4488590B8A3AA73927FE5, ordinal: 0
2016-01-14 16:29:47,433 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BCA6E44ACBC4488590B8A3AA73927FE5, mem on start (mb), free: 160, total: 461, max: 672
2016-01-14 16:29:47,433 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:47,434 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:47,434 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:47,440 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 16:29:47,440 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 9D68F129978F402299C369E042841A38
2016-01-14 16:29:47,440 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:47,440 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:47,440 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 16:29:47,441 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:47,441 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 16:29:47,441 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:47,441 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 16:29:47,441 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-01-14 16:29:47,441 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A580F15BA58443DE826FAB4BD2B077A9, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 16:29:47,441 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 9D68F129978F402299C369E042841A38, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 16:29:47,441 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:47,442 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A580F15BA58443DE826FAB4BD2B077A9 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:47,442 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A580F15BA58443DE826FAB4BD2B077A9
2016-01-14 16:29:47,442 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:47,442 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A580F15BA58443DE826FAB4BD2B077A9
2016-01-14 16:29:47,442 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 16:29:47,443 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:47,443 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:47,443 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788987185_0001_1_00_000000_0
2016-01-14 16:29:47,443 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 16:29:47,445 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@39714859
2016-01-14 16:29:47,447 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@39714859
2016-01-14 16:29:47,447 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 16:29:47,447 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 16:29:47,447 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower+upper.txt"]
2016-01-14 16:29:47,449 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower+upper.txt"] streamed: true, id: A580F15BA58443DE826FAB4BD2B077A9
2016-01-14 16:29:47,449 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped"], id: 9D68F129978F402299C369E042841A38
2016-01-14 16:29:47,450 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped"]
2016-01-14 16:29:47,450 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower+upper.txt"]
2016-01-14 16:29:47,450 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:47,451 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BCA6E44ACBC4488590B8A3AA73927FE5, all 1 inputs ready in: 00:00:00.000
2016-01-14 16:29:47,454 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 16:29:47,454 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 16:29:47,455 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452788987185_0001_1_00_000000_0
2016-01-14 16:29:47,455 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452788987185_0001_1_00_000000_0 given a go for committing the task output.
2016-01-14 16:29:47,456 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788987185_0001_1_00_000000_0
2016-01-14 16:29:47,456 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BCA6E44ACBC4488590B8A3AA73927FE5, mem on close (mb), free: 157, total: 461, max: 672
2016-01-14 16:29:47 Completed running task attempt: attempt_1452788987185_0001_1_00_000000_0
2016-01-14 16:29:47,457 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788987185_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:47,458 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10][cascading.flow.SliceCounters Process_Begin_Time=1452788987451, Process_Duration=5, Process_End_Time=1452788987456, Read_Duration=1, Tuples_Read=5, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5, Tuples_Written=10]]
2016-01-14 16:29:47,458 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:47,458 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:47,458 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788987185_0001_00_000001
2016-01-14 16:29:47,458 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987185_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BCA6E44ACBC4488590B8A3AA73927FE5, taskAttemptId=attempt_1452788987185_0001_1_00_000000_0, startTime=1452788987408, finishTime=1452788987458, timeTaken=50, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452788987451, Process_Duration=5, Process_End_Time=1452788987456, Read_Duration=1, Tuples_Read=5, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10
2016-01-14 16:29:47,458 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:47,458 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788987185_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:47,459 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:47,459 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788987185_0001_1_00_000000_0
2016-01-14 16:29:47,460 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:47,460 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987185_0001_1][Event:TASK_FINISHED]: vertexName=BCA6E44ACBC4488590B8A3AA73927FE5, taskId=task_1452788987185_0001_1_00_000000, startTime=1452788987408, finishTime=1452788987459, timeTaken=51, status=SUCCEEDED, successfulAttemptID=attempt_1452788987185_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452788987451, Process_Duration=5, Process_End_Time=1452788987456, Read_Duration=1, Tuples_Read=5, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10
2016-01-14 16:29:47,461 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788987185_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:47,461 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788987185_0001_1_00 [BCA6E44ACBC4488590B8A3AA73927FE5] : 1
2016-01-14 16:29:47,461 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788987185_0001_1_00 [BCA6E44ACBC4488590B8A3AA73927FE5]
2016-01-14 16:29:47,464 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:47,464 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987185_0001_1][Event:VERTEX_FINISHED]: vertexName=BCA6E44ACBC4488590B8A3AA73927FE5, vertexId=vertex_1452788987185_0001_1_00, initRequestedTime=1452788987378, initedTime=1452788987393, startRequestedTime=1452788987395, startedTime=1452788987395, finishTime=1452788987461, timeTaken=66, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452788987451, Process_Duration=5, Process_End_Time=1452788987456, Read_Duration=1, Tuples_Read=5, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10, vertexStats=firstTaskStartTime=1452788987408, firstTasksToStart=[ task_1452788987185_0001_1_00_000000 ], lastTaskFinishTime=1452788987458, lastTasksToFinish=[ task_1452788987185_0001_1_00_000000 ], minTaskDuration=50, maxTaskDuration=50, avgTaskDuration=50.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788987185_0001_1_00_000000 ], longestDurationTasks=[ task_1452788987185_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:47,465 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788987185_0001_1_00 [BCA6E44ACBC4488590B8A3AA73927FE5] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:47,465 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788987185_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:47,465 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987185_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788987185_0001_00_000001, stoppedTime=1452788987465, exitStatus=0
2016-01-14 16:29:47,466 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788987185_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:47,466 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788987185_0001_1_00 [BCA6E44ACBC4488590B8A3AA73927FE5] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-01-14 16:29:47,467 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452788987185_0001_1
2016-01-14 16:29:47,465 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:47,467 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987185_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452788987185_0001_1
2016-01-14 16:29:47,467 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 9D68F129978F402299C369E042841A38 for vertex: vertex_1452788987185_0001_1_00 [BCA6E44ACBC4488590B8A3AA73927FE5]
2016-01-14 16:29:47,470 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452788987185_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 16:29:47,470 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788987185_0001_00_000001: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 16:29:47,470 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452788987185_0001_00_000001, Exiting
2016-01-14 16:29:47,471 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788987185_0001_00_000001
2016-01-14 16:29:47,471 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788987185_0001_00_000001
2016-01-14 16:29:47,474 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788987185_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 16:29:47,474 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452788987185_0001_00_000001 completed successfully
2016-01-14 16:29:47,480 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452788987185_0001_1, queueSize=0
2016-01-14 16:29:47,480 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 16:29:47,481 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987185_0001_1][Event:DAG_FINISHED]: dagId=dag_1452788987185_0001_1, startTime=1452788987378, finishTime=1452788987479, timeTaken=101, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452788987451, Process_Duration=5, Process_End_Time=1452788987456, Read_Duration=1, Tuples_Read=5, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10
2016-01-14 16:29:47,481 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452788987185_0001_1 finished with state: SUCCEEDED
2016-01-14 16:29:47,481 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788987185_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 16:29:47,482 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788987185_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 16:29:47,482 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452788987185_0001_00_000001
2016-01-14 16:29:47 Completed Dag: dag_1452788987185_0001_1
2016-01-14 16:29:47,483 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452788987185_0001_1, dagState=SUCCEEDED
2016-01-14 16:29:47,484 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788987185_0001_1_post
2016-01-14 16:29:47,484 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:47,485 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 16:29:47,488 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...orm/testUnGroup/ungrouped, applicationId=application_1452788987185_0001
2016-01-14 16:29:47,491 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 16:29:47,491 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 16:29:47,491 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 16:29:47,491 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 16:29:47,491 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 16:29:47,492 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 16:29:47,492 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 16:29:47,493 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 16:29:47,493 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 16:29:47,493 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 16:29:47,494 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 16:29:47,495 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 16:29:47,495 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 16:29:47,500 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/FC740EBBE19E42FAA54FC3C93434CADE/AC645A05C347467FB0134583CE34BDB5/.tez/application_1452788987185_0001
2016-01-14 16:29:47,501 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:00.337
2016-01-14 16:29:47,539 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped
2016-01-14 16:29:47,574 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 16:29:47,574 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 16:29:47,628 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.026
2016-01-14 16:29:47,643 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.042
2016-01-14 16:29:47,644 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-01-14 16:29:47,644 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-01-14 16:29:47,645 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 16:29:47,666 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt
2016-01-14 16:29:47,679 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt
2016-01-14 16:29:47,690 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown
2016-01-14 16:29:47,761 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] starting
2016-01-14 16:29:47,762 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-01-14 16:29:47,762 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 16:29:47,762 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown"]
2016-01-14 16:29:47,762 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] parallel execution of steps is enabled: false
2016-01-14 16:29:47,762 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] executing total steps: 1
2016-01-14 16:29:47,762 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] allocating management threads: 1
2016-01-14 16:29:47,763 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] starting step: (1/1) ...lexcogrouprelativeunknown
2016-01-14 16:29:47,763 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [lower*upper] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 16:29:47,787 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] tez session mode enabled: true
2016-01-14 16:29:47,793 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:47,794 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 16:29:47,794 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 16:29:47,804 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1B3AC423F00B45168A717F7CD97E3F89/D54D3F0F6EFF4E51B34B622B3C6DA7B2/.tez/application_1452788987794_0001 doesn't exist and is created
2016-01-14 16:29:47,823 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 16:29:47,823 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 16:29:47,824 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1B3AC423F00B45168A717F7CD97E3F89/D54D3F0F6EFF4E51B34B622B3C6DA7B2/.tez/application_1452788987794_0001
2016-01-14 16:29:47,828 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452788987794_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:47,842 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 16:29:47,844 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 16:29:47,846 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 16:29:47,846 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 16:29:47,846 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1B3AC423F00B45168A717F7CD97E3F89/D54D3F0F6EFF4E51B34B622B3C6DA7B2/.tez/application_1452788987794_0001/localmode-log-dir
2016-01-14 16:29:47,846 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1B3AC423F00B45168A717F7CD97E3F89/D54D3F0F6EFF4E51B34B622B3C6DA7B2/.tez/application_1452788987794_0001/localmode-log-dir/history.txt.appattempt_1452788987794_0001_000000, maxErrors=10
2016-01-14 16:29:47,847 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 16:29:47,847 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452788987794_0001_000000, appSubmitTime=1452788987827, launchTime=1452788987827
2016-01-14 16:29:47,852 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 16:29:47,854 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 16:29:47,856 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 16:29:47,856 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 16:29:47,863 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:60728
2016-01-14 16:29:47,864 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 16:29:47,868 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 16:29:47,868 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452788987794_0001_000000, startTime=1452788987867
2016-01-14 16:29:47,868 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 16:29:47,923 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 16:29:47,924 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 16:29:47,924 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lexcogrouprelativeunknown, applicationId=application_1452788987794_0001, dagName=[1B3AC423F00B45168A717F7CD97E3F89/D54D3F0F6EFF4E51B34B622B3C6DA7B2] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-01-14 16:29:47,943 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [1B3AC423F00B45168A717F7CD97E3F89/D54D3F0F6EFF4E51B34B622B3C6DA7B2] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-01-14 16:29:47,945 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452788987794_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1B3AC423F00B45168A717F7CD97E3F89/D54D3F0F6EFF4E51B34B622B3C6DA7B2/.tez/application_1452788987794_0001/localmode-log-dir/dag_1452788987794_0001_1.dot
2016-01-14 16:29:47,948 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1B3AC423F00B45168A717F7CD97E3F89/D54D3F0F6EFF4E51B34B622B3C6DA7B2/.tez/application_1452788987794_0001/localmode-log-dir/dag_1452788987794_0001_1-tez-dag.pb.txt
2016-01-14 16:29:47,974 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788987794_0001_1
2016-01-14 16:29:47,975 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:47,975 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [1B3AC423F00B45168A717F7CD97E3F89/D54D3F0F6EFF4E51B34B622B3C6DA7B2] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-01-14 16:29:47 Running Dag: dag_1452788987794_0001_1
2016-01-14 16:29:47,992 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452788987794_0001_1, submitTime=1452788987943
2016-01-14 16:29:47,992 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 16:29:47,993 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F2A2F6D6AF4347EB95C68B0EB54535A0
2016-01-14 16:29:47,993 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 16:29:47,995 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452788987794_0001_1, initTime=1452788987992
2016-01-14 16:29:47,995 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 16:29:47,995 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788987794_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 16:29:47,996 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:DAG_STARTED]: dagID=dag_1452788987794_0001_1, startTime=1452788987996
2016-01-14 16:29:47,996 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 16:29:47,997 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788987794_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 16:29:47,997 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C134198274F146E7A28244CFC6E2A96F : {3D6D00D1030349CFB409C46A739361F7={InputName=3D6D00D1030349CFB409C46A739361F7}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 16:29:47,997 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 3D6D00D1030349CFB409C46A739361F7, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 16:29:47,997 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452788987794_0001_1_00 [C134198274F146E7A28244CFC6E2A96F]
2016-01-14 16:29:47,997 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452788987794_0001_1_00 [C134198274F146E7A28244CFC6E2A96F]
2016-01-14 16:29:47,997 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452788987794_0001_1_00 [C134198274F146E7A28244CFC6E2A96F]
2016-01-14 16:29:47,997 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452788987794_0001_1_00 [C134198274F146E7A28244CFC6E2A96F]. Starting root input initializers: 1
2016-01-14 16:29:47,996 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 16:29:47,997 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lexcogrouprelativeunknown, applicationId=application_1452788987794_0001, dagName=[1B3AC423F00B45168A717F7CD97E3F89/D54D3F0F6EFF4E51B34B622B3C6DA7B2] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-01-14 16:29:47,999 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788987794_0001_1_00 [C134198274F146E7A28244CFC6E2A96F] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 16:29:47,999 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C656C04D1FD84351A746A11A4B11CD6E : {69884D7973FF4D388726D9C008D22F21={InputName=69884D7973FF4D388726D9C008D22F21}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 16:29:47,999 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 69884D7973FF4D388726D9C008D22F21, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 16:29:47,999 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452788987794_0001_1_01 [C656C04D1FD84351A746A11A4B11CD6E]
2016-01-14 16:29:47,999 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452788987794_0001_1_01 [C656C04D1FD84351A746A11A4B11CD6E]
2016-01-14 16:29:47,999 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452788987794_0001_1_01 [C656C04D1FD84351A746A11A4B11CD6E]
2016-01-14 16:29:48,000 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452788987794_0001_1_01 [C656C04D1FD84351A746A11A4B11CD6E]. Starting root input initializers: 1
2016-01-14 16:29:48,000 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 3D6D00D1030349CFB409C46A739361F7 on vertex vertex_1452788987794_0001_1_00 [C134198274F146E7A28244CFC6E2A96F]
2016-01-14 16:29:48,001 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788987794_0001_1_01 [C656C04D1FD84351A746A11A4B11CD6E] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 16:29:48,001 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452788987794_0001_1_02 [F2A2F6D6AF4347EB95C68B0EB54535A0]
2016-01-14 16:29:48,002 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 69884D7973FF4D388726D9C008D22F21 on vertex vertex_1452788987794_0001_1_01 [C656C04D1FD84351A746A11A4B11CD6E]
2016-01-14 16:29:48,006 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 16:29:48,007 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 3D6D00D1030349CFB409C46A739361F7 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 16:29:48,008 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 16:29:48,008 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 16:29:48,010 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 16:29:48,011 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 69884D7973FF4D388726D9C008D22F21 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 16:29:48,012 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-01-14 16:29:48,013 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 3D6D00D1030349CFB409C46A739361F7 on vertex vertex_1452788987794_0001_1_00 [C134198274F146E7A28244CFC6E2A96F]
2016-01-14 16:29:48,012 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 16:29:48,013 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452788987794_0001_1_02 [F2A2F6D6AF4347EB95C68B0EB54535A0]
2016-01-14 16:29:48,014 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452788987794_0001_1_02 [F2A2F6D6AF4347EB95C68B0EB54535A0]
2016-01-14 16:29:48,014 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452788987794_0001_1_02 [F2A2F6D6AF4347EB95C68B0EB54535A0]
2016-01-14 16:29:48,014 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=96E1F871FA5243C9861E599B4477FA65, vertexId=vertex_1452788987794_0001_1_02 [F2A2F6D6AF4347EB95C68B0EB54535A0], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 16:29:48,014 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=96E1F871FA5243C9861E599B4477FA65, vertexId=vertex_1452788987794_0001_1_02 [F2A2F6D6AF4347EB95C68B0EB54535A0]
2016-01-14 16:29:48,018 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 16:29:48,018 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=96E1F871FA5243C9861E599B4477FA65, vertexName=F2A2F6D6AF4347EB95C68B0EB54535A0, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 16:29:48,018 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=96E1F871FA5243C9861E599B4477FA65, vertexId=vertex_1452788987794_0001_1_02 [F2A2F6D6AF4347EB95C68B0EB54535A0]
2016-01-14 16:29:48,020 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 16:29:48,022 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-01-14 16:29:48,023 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:48,023 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] submitted tez dag to app master: application_1452788987794_0001, with dag id: dag_1452788987794_0001_1
2016-01-14 16:29:48,023 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F2A2F6D6AF4347EB95C68B0EB54535A0, vertexId=vertex_1452788987794_0001_1_02, initRequestedTime=1452788988001, initedTime=1452788988020, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 16:29:48,024 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788987794_0001_1_02 [F2A2F6D6AF4347EB95C68B0EB54535A0] transitioned from NEW to INITED due to event V_INIT
2016-01-14 16:29:48,025 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788987794_0001_1_00 [C134198274F146E7A28244CFC6E2A96F] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:48,025 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {3D6D00D1030349CFB409C46A739361F7=forAllWorkUnits=true, update=[1]}
2016-01-14 16:29:48,025 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452788987794_0001_1_00 [C134198274F146E7A28244CFC6E2A96F] parallelism set to 1
2016-01-14 16:29:48,025 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788987794_0001_1_00 [C134198274F146E7A28244CFC6E2A96F] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:48,026 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788987794_0001_1_00, eventCount=1
2016-01-14 16:29:48,026 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:48,027 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C134198274F146E7A28244CFC6E2A96F, vertexId=vertex_1452788987794_0001_1_00, initRequestedTime=1452788987997, initedTime=1452788988026, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 16:29:48,027 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788987794_0001_1_00 [C134198274F146E7A28244CFC6E2A96F] with distanceFromRoot: 0
2016-01-14 16:29:48,027 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788987794_0001_1_00 [C134198274F146E7A28244CFC6E2A96F] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 16:29:48,028 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C134198274F146E7A28244CFC6E2A96F
2016-01-14 16:29:48,028 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788987794_0001_1_00 [C134198274F146E7A28244CFC6E2A96F]
2016-01-14 16:29:48,028 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:48,029 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788987794_0001_1_00, startRequestedTime=1452788988028, startedTime=1452788988028
2016-01-14 16:29:48,029 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788987794_0001_1_00 [C134198274F146E7A28244CFC6E2A96F] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:48,030 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:48,030 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:TASK_STARTED]: vertexName=C134198274F146E7A28244CFC6E2A96F, taskId=task_1452788987794_0001_1_00_000000, scheduledTime=1452788988029, launchTime=1452788988029
2016-01-14 16:29:48,030 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788987794_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:48,031 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452788987794_0001_1_00 for vertex: vertex_1452788987794_0001_1_02 [F2A2F6D6AF4347EB95C68B0EB54535A0] numStartedSources: 1 numSources: 2
2016-01-14 16:29:48,031 INFO impl.VertexImpl (VertexImpl.java:transition(3160)) - Cannot start vertex: vertex_1452788987794_0001_1_02 [F2A2F6D6AF4347EB95C68B0EB54535A0] numStartedSources: 1 numSources: 2
2016-01-14 16:29:48,031 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788987794_0001_1_00 [C134198274F146E7A28244CFC6E2A96F]
2016-01-14 16:29:48,033 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1B3AC423F00B45168A717F7CD97E3F89/D54D3F0F6EFF4E51B34B622B3C6DA7B2] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: C134198274F146E7A28244CFC6E2A96F, VertexParallelism: 1, TaskAttemptID:attempt_1452788987794_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3D6D00D1030349CFB409C46A739361F7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F2A2F6D6AF4347EB95C68B0EB54535A0, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 16:29:48,034 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788987794_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:48,034 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 69884D7973FF4D388726D9C008D22F21 on vertex vertex_1452788987794_0001_1_01 [C656C04D1FD84351A746A11A4B11CD6E]
2016-01-14 16:29:48,034 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:48,041 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788987794_0001_1_01 [C656C04D1FD84351A746A11A4B11CD6E] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:48,041 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {69884D7973FF4D388726D9C008D22F21=forAllWorkUnits=true, update=[1]}
2016-01-14 16:29:48,042 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452788987794_0001_1_01 [C656C04D1FD84351A746A11A4B11CD6E] parallelism set to 1
2016-01-14 16:29:48,042 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788987794_0001_1_01 [C656C04D1FD84351A746A11A4B11CD6E] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:48,043 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 16:29:48,043 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788987794_0001_1_01, eventCount=1
2016-01-14 16:29:48,044 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:48,044 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C656C04D1FD84351A746A11A4B11CD6E, vertexId=vertex_1452788987794_0001_1_01, initRequestedTime=1452788987999, initedTime=1452788988044, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 16:29:48,045 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788987794_0001_1_01 [C656C04D1FD84351A746A11A4B11CD6E] with distanceFromRoot: 0
2016-01-14 16:29:48,045 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788987794_0001_1_01 [C656C04D1FD84351A746A11A4B11CD6E] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 16:29:48,046 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 16:29:48,047 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 16:29:48,049 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788987794_0001_00_000001
2016-01-14 16:29:48,050 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788987794_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:48,050 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C656C04D1FD84351A746A11A4B11CD6E
2016-01-14 16:29:48,051 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788987794_0001_1_01 [C656C04D1FD84351A746A11A4B11CD6E]
2016-01-14 16:29:48,051 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788987794_0001_1_01, startRequestedTime=1452788988050, startedTime=1452788988050
2016-01-14 16:29:48,051 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:48,051 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788987794_0001_1_01 [C656C04D1FD84351A746A11A4B11CD6E] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:48,052 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:48,052 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:TASK_STARTED]: vertexName=C656C04D1FD84351A746A11A4B11CD6E, taskId=task_1452788987794_0001_1_01_000000, scheduledTime=1452788988052, launchTime=1452788988052
2016-01-14 16:29:48,053 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788987794_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:48,053 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452788987794_0001_1_01 for vertex: vertex_1452788987794_0001_1_02 [F2A2F6D6AF4347EB95C68B0EB54535A0] numStartedSources: 2 numSources: 2
2016-01-14 16:29:48,054 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788987794_0001_1_02 [F2A2F6D6AF4347EB95C68B0EB54535A0] with distanceFromRoot: 1
2016-01-14 16:29:48,054 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788987794_0001_1_01 [C656C04D1FD84351A746A11A4B11CD6E]
2016-01-14 16:29:48,054 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788987794_0001_00_000001
2016-01-14 16:29:48,055 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 16:29:48,055 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C656C04D1FD84351A746A11A4B11CD6E in vertex: F2A2F6D6AF4347EB95C68B0EB54535A0
2016-01-14 16:29:48,056 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C134198274F146E7A28244CFC6E2A96F in vertex: F2A2F6D6AF4347EB95C68B0EB54535A0
2016-01-14 16:29:48,057 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F2A2F6D6AF4347EB95C68B0EB54535A0 with 2 source tasks and 1 pending tasks
2016-01-14 16:29:48,056 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788987794_0001_00_000001, launchTime=1452788988056
2016-01-14 16:29:48,056 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:48,058 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788987794_0001_00_000001
2016-01-14 16:29:48,057 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788987794_0001_1_02, startRequestedTime=1452788988055, startedTime=1452788988055
2016-01-14 16:29:48,058 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:48,058 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788987794_0001_1_02 [F2A2F6D6AF4347EB95C68B0EB54535A0] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:48,059 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1B3AC423F00B45168A717F7CD97E3F89/D54D3F0F6EFF4E51B34B622B3C6DA7B2] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: C656C04D1FD84351A746A11A4B11CD6E, VertexParallelism: 1, TaskAttemptID:attempt_1452788987794_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=69884D7973FF4D388726D9C008D22F21, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F2A2F6D6AF4347EB95C68B0EB54535A0, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 16:29:48,060 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788987794_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:48,060 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788987794_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:48,060 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:48,062 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788987794_0001_00_000002
2016-01-14 16:29:48,062 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788987794_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:48,064 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788987794_0001_00_000002
2016-01-14 16:29:48,065 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 16:29:48,065 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788987794_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:48,065 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:48,066 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788987794_0001_00_000002, launchTime=1452788988065
2016-01-14 16:29:48,067 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788987794_0001_00_000002
2016-01-14 16:29:48,068 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788987794_0001_1_01_000000_0] to container: [container_1452788987794_0001_00_000002]
2016-01-14 16:29:48,068 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788987794_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:48,068 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788987794_0001_00_000002 given task: attempt_1452788987794_0001_1_01_000000_0
2016-01-14 16:29:48,068 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788987794_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788987794_0001_1_01_000000_0
2016-01-14 16:29:48,069 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788987794_0001_1_00_000000_0] to container: [container_1452788987794_0001_00_000001]
2016-01-14 16:29:48,070 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788987794_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:48,070 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788987794_0001_1_01_000000_0] started. Is using containerId: [container_1452788987794_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 16:29:48,070 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788987794_0001_00_000001 given task: attempt_1452788987794_0001_1_00_000000_0
2016-01-14 16:29:48,070 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788987794_0001_00_000001: 12 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788987794_0001_1_00_000000_0
2016-01-14 16:29:48,070 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:48 Starting to run new task attempt: attempt_1452788987794_0001_1_00_000000_0
2016-01-14 16:29:48,071 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C656C04D1FD84351A746A11A4B11CD6E, taskAttemptId=attempt_1452788987794_0001_1_01_000000_0, startTime=1452788988068, containerId=container_1452788987794_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788987794_0001_00_000002/teamcity, completedLogs=
2016-01-14 16:29:48,072 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788987794_0001_1_00_000000_0
2016-01-14 16:29:48,073 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:48,073 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:48,073 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:48,073 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:48,074 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:48,074 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:48,074 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1B3AC423F00B45168A717F7CD97E3F89/D54D3F0F6EFF4E51B34B622B3C6DA7B2] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: C134198274F146E7A28244CFC6E2A96F, VertexParallelism: 1, TaskAttemptID:attempt_1452788987794_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3D6D00D1030349CFB409C46A739361F7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F2A2F6D6AF4347EB95C68B0EB54535A0, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 16:29:48,074 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:48,076 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788987794_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:48,077 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788987794_0001_1_00_000000_0] started. Is using containerId: [container_1452788987794_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 16:29:48,078 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:48,078 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C134198274F146E7A28244CFC6E2A96F, taskAttemptId=attempt_1452788987794_0001_1_00_000000_0, startTime=1452788988077, containerId=container_1452788987794_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788987794_0001_00_000001/teamcity, completedLogs=
2016-01-14 16:29:48 Starting to run new task attempt: attempt_1452788987794_0001_1_01_000000_0
2016-01-14 16:29:48,079 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788987794_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:48,079 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788987794_0001_1_01_000000_0
2016-01-14 16:29:48,080 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:48,080 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:48,080 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:48,080 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788987794_0001_1_00_000000_0
2016-01-14 16:29:48,080 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:48,081 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:48,080 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788987794_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:48,081 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788987794_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:48,082 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:48,080 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:48,082 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:48,081 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=3D6D00D1030349CFB409C46A739361F7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 16:29:48,083 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:48,083 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 3D6D00D1030349CFB409C46A739361F7
2016-01-14 16:29:48,084 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 3D6D00D1030349CFB409C46A739361F7
2016-01-14 16:29:48,086 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:48,088 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1B3AC423F00B45168A717F7CD97E3F89/D54D3F0F6EFF4E51B34B622B3C6DA7B2] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: C656C04D1FD84351A746A11A4B11CD6E, VertexParallelism: 1, TaskAttemptID:attempt_1452788987794_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=69884D7973FF4D388726D9C008D22F21, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F2A2F6D6AF4347EB95C68B0EB54535A0, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 16:29:48,088 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:48,086 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788987794_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 16:29:48,090 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 16:29:48,092 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 3D6D00D1030349CFB409C46A739361F7
2016-01-14 16:29:48,092 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 3D6D00D1030349CFB409C46A739361F7
2016-01-14 16:29:48,089 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F2A2F6D6AF4347EB95C68B0EB54535A0, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 16:29:48,094 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:48,094 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F2A2F6D6AF4347EB95C68B0EB54535A0
2016-01-14 16:29:48,096 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:48,097 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 16:29:48,097 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F2A2F6D6AF4347EB95C68B0EB54535A0
2016-01-14 16:29:48,099 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F2A2F6D6AF4347EB95C68B0EB54535A0
2016-01-14 16:29:48,099 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788987794_0001_1_01_000000_0
2016-01-14 16:29:48,100 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:48,100 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:48,100 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788987794_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 16:29:48,101 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=69884D7973FF4D388726D9C008D22F21, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 16:29:48,101 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:48,102 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 69884D7973FF4D388726D9C008D22F21
2016-01-14 16:29:48,102 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 69884D7973FF4D388726D9C008D22F21
2016-01-14 16:29:48,104 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:48,108 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C134198274F146E7A28244CFC6E2A96F, ordinal: 1
2016-01-14 16:29:48,108 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C134198274F146E7A28244CFC6E2A96F, mem on start (mb), free: 50, total: 461, max: 672
2016-01-14 16:29:48,108 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:48,108 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:48,109 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:48,109 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:48,109 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:48,110 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 16:29:48,110 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:48,114 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F2A2F6D6AF4347EB95C68B0EB54535A0, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 16:29:48,114 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:48,115 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F2A2F6D6AF4347EB95C68B0EB54535A0
2016-01-14 16:29:48,117 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:48,117 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 16:29:48,118 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 16:29:48,119 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 16:29:48,119 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 69884D7973FF4D388726D9C008D22F21
2016-01-14 16:29:48,120 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 16:29:48,120 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 3D6D00D1030349CFB409C46A739361F7, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 16:29:48,120 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F2A2F6D6AF4347EB95C68B0EB54535A0, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 16:29:48,122 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:48,122 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 3D6D00D1030349CFB409C46A739361F7 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:48,122 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 16:29:48,122 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:48,120 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 69884D7973FF4D388726D9C008D22F21
2016-01-14 16:29:48,122 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C656C04D1FD84351A746A11A4B11CD6E, ordinal: 0
2016-01-14 16:29:48,122 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 3D6D00D1030349CFB409C46A739361F7
2016-01-14 16:29:48,123 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:48,123 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 3D6D00D1030349CFB409C46A739361F7
2016-01-14 16:29:48,121 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 16:29:48,123 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F2A2F6D6AF4347EB95C68B0EB54535A0
2016-01-14 16:29:48,123 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:48,123 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C656C04D1FD84351A746A11A4B11CD6E, mem on start (mb), free: 49, total: 461, max: 672
2016-01-14 16:29:48,124 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788987794_0001_1_00_000000_0
2016-01-14 16:29:48,124 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F2A2F6D6AF4347EB95C68B0EB54535A0
2016-01-14 16:29:48,125 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 16:29:48,125 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@34320932
2016-01-14 16:29:48,127 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@34320932
2016-01-14 16:29:48,127 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 16:29:48,127 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 16:29:48,127 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:48,128 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:48,128 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 16:29:48,128 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:48,129 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:48,129 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:48,132 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"] streamed: true, id: 3D6D00D1030349CFB409C46A739361F7
2016-01-14 16:29:48,133 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]], id: 0C2AECB450894ED3BFCC943F94A258B7
2016-01-14 16:29:48,133 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] 0C2AECB450894ED3BFCC943F94A258B7
2016-01-14 16:29:48,133 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 16:29:48,134 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:48,135 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 16:29:48,135 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 16:29:48,135 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 16:29:48,135 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 69884D7973FF4D388726D9C008D22F21, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 16:29:48,136 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F2A2F6D6AF4347EB95C68B0EB54535A0, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 16:29:48,136 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:48,136 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 69884D7973FF4D388726D9C008D22F21 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:48,136 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 69884D7973FF4D388726D9C008D22F21
2016-01-14 16:29:48,136 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:48,136 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 69884D7973FF4D388726D9C008D22F21
2016-01-14 16:29:48,136 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@7824d24b; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@396a778c; comparator=cascading.tuple.hadoop.util.TupleComparator@7ccea5c6; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 16:29:48,138 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 16:29:48,138 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 16:29:48,138 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 16:29:48,164 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 16:29:48,164 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:48,164 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:48,165 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788987794_0001_1_01_000000_0
2016-01-14 16:29:48,165 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 16:29:48,168 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@654e75ca
2016-01-14 16:29:48,170 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@654e75ca
2016-01-14 16:29:48,170 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 16:29:48,170 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 16:29:48,180 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 16:29:48,180 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 16:29:48,180 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 16:29:48,181 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 16:29:48,181 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 16:29:48,180 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-01-14 16:29:48,182 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 16:29:48,182 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:48,182 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C134198274F146E7A28244CFC6E2A96F, all 1 inputs ready in: 00:00:00.000
2016-01-14 16:29:48,182 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt"] streamed: true, id: 69884D7973FF4D388726D9C008D22F21
2016-01-14 16:29:48,183 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]], id: 0C2AECB450894ED3BFCC943F94A258B7
2016-01-14 16:29:48,183 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] 0C2AECB450894ED3BFCC943F94A258B7
2016-01-14 16:29:48,184 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@373d3210; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@12a1d4c5; comparator=cascading.tuple.hadoop.util.TupleComparator@3e97db11; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 16:29:48,186 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 16:29:48,186 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 16:29:48,186 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 16:29:48,187 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788987794_0001_1_00_000000_0
2016-01-14 16:29:48,188 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C134198274F146E7A28244CFC6E2A96F, mem on close (mb), free: 40, total: 441, max: 672
2016-01-14 16:29:48,188 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 16:29:48,188 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 16:29:48,188 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-01-14 16:29:48,188 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-01-14 16:29:48,193 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 16:29:48,204 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 16:29:48,205 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 16:29:48,205 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 16:29:48,205 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 16:29:48,205 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 16:29:48 Completed running task attempt: attempt_1452788987794_0001_1_00_000000_0
2016-01-14 16:29:48,206 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-01-14 16:29:48,206 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:48,206 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788987794_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:48,206 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C656C04D1FD84351A746A11A4B11CD6E, all 1 inputs ready in: 00:00:00.000
2016-01-14 16:29:48,210 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788987794_0001_1_00, eventCount=1
2016-01-14 16:29:48,210 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=25, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=462422016, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452788988182, Process_Duration=5, Process_End_Time=1452788988187, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-01-14 16:29:48,210 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:48,210 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:48,211 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788987794_0001_00_000001
2016-01-14 16:29:48,211 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C134198274F146E7A28244CFC6E2A96F, taskAttemptId=attempt_1452788987794_0001_1_00_000000_0, startTime=1452788988077, finishTime=1452788988210, timeTaken=133, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=25, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=462422016, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788988182, Process_Duration=5, Process_End_Time=1452788988187, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 16:29:48,211 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788987794_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:48,211 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788987794_0001_1_00_000000_0
2016-01-14 16:29:48,212 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:48,212 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:TASK_FINISHED]: vertexName=C134198274F146E7A28244CFC6E2A96F, taskId=task_1452788987794_0001_1_00_000000, startTime=1452788988077, finishTime=1452788988211, timeTaken=134, status=SUCCEEDED, successfulAttemptID=attempt_1452788987794_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=25, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=462422016, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788988182, Process_Duration=5, Process_End_Time=1452788988187, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 16:29:48,212 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788987794_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:48,213 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:48,213 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788987794_0001_1_00 [C134198274F146E7A28244CFC6E2A96F] : 1
2016-01-14 16:29:48,213 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788987794_0001_1_00 [C134198274F146E7A28244CFC6E2A96F]
2016-01-14 16:29:48,214 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:48,215 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788987794_0001_1_01_000000_0
2016-01-14 16:29:48,215 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C656C04D1FD84351A746A11A4B11CD6E, mem on close (mb), free: 37, total: 441, max: 672
2016-01-14 16:29:48,216 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 16:29:48,216 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 16:29:48,216 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-01-14 16:29:48,216 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-01-14 16:29:48,218 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:48,219 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:VERTEX_FINISHED]: vertexName=C134198274F146E7A28244CFC6E2A96F, vertexId=vertex_1452788987794_0001_1_00, initRequestedTime=1452788987997, initedTime=1452788988026, startRequestedTime=1452788988028, startedTime=1452788988028, finishTime=1452788988213, timeTaken=185, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=25, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=462422016, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788988182, Process_Duration=5, Process_End_Time=1452788988187, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1452788988077, firstTasksToStart=[ task_1452788987794_0001_1_00_000000 ], lastTaskFinishTime=1452788988210, lastTasksToFinish=[ task_1452788987794_0001_1_00_000000 ], minTaskDuration=133, maxTaskDuration=133, avgTaskDuration=133.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788987794_0001_1_00_000000 ], longestDurationTasks=[ task_1452788987794_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:48,220 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788987794_0001_1_00 [C134198274F146E7A28244CFC6E2A96F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:48,220 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788987794_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:48,221 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788987794_0001_00_000001, stoppedTime=1452788988221, exitStatus=0
2016-01-14 16:29:48,221 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:48,221 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788987794_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:48,222 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452788987794_0001_1_02 [F2A2F6D6AF4347EB95C68B0EB54535A0] attempt: attempt_1452788987794_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 16:29:48,222 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788987794_0001_1_00 [C134198274F146E7A28244CFC6E2A96F] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 16:29:48,223 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 16:29:48,223 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452788987794_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 16:29:48,224 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788987794_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 16:29:48,223 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452788987794_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 16:29:48,225 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788987794_0001_00_000001: 14 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 16:29:48,225 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452788987794_0001_00_000001, Exiting
2016-01-14 16:29:48,225 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788987794_0001_00_000001
2016-01-14 16:29:48,226 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788987794_0001_00_000001
2016-01-14 16:29:48,233 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788987794_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 16:29:48,233 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452788987794_0001_00_000001 completed successfully
2016-01-14 16:29:48,233 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788987794_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 16:29:48 Completed running task attempt: attempt_1452788987794_0001_1_01_000000_0
2016-01-14 16:29:48,236 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788987794_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:48,237 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=8155, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=25, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=462422016, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452788988207, Process_Duration=8, Process_End_Time=1452788988215, Read_Duration=6, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-01-14 16:29:48,238 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:48,238 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:48,238 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788987794_0001_00_000002
2016-01-14 16:29:48,237 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788987794_0001_1_01, eventCount=1
2016-01-14 16:29:48,239 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:48,240 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C656C04D1FD84351A746A11A4B11CD6E, taskAttemptId=attempt_1452788987794_0001_1_01_000000_0, startTime=1452788988068, finishTime=1452788988239, timeTaken=171, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=8155, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=25, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=462422016, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788988207, Process_Duration=8, Process_End_Time=1452788988215, Read_Duration=6, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 16:29:48,240 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788987794_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:48,240 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:48,240 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788987794_0001_1_01_000000_0
2016-01-14 16:29:48,241 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:48,241 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:TASK_FINISHED]: vertexName=C656C04D1FD84351A746A11A4B11CD6E, taskId=task_1452788987794_0001_1_01_000000, startTime=1452788988068, finishTime=1452788988241, timeTaken=173, status=SUCCEEDED, successfulAttemptID=attempt_1452788987794_0001_1_01_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=8155, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=25, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=462422016, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788988207, Process_Duration=8, Process_End_Time=1452788988215, Read_Duration=6, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 16:29:48,242 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788987794_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:48,242 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788987794_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:48,242 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:48,242 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788987794_0001_00_000002, stoppedTime=1452788988242, exitStatus=0
2016-01-14 16:29:48,243 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788987794_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:48,243 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788987794_0001_1_01 [C656C04D1FD84351A746A11A4B11CD6E] : 1
2016-01-14 16:29:48,243 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788987794_0001_1_01 [C656C04D1FD84351A746A11A4B11CD6E]
2016-01-14 16:29:48,244 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:48,250 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452788987794_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 16:29:48,251 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788987794_0001_00_000002: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 16:29:48,251 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:VERTEX_FINISHED]: vertexName=C656C04D1FD84351A746A11A4B11CD6E, vertexId=vertex_1452788987794_0001_1_01, initRequestedTime=1452788987999, initedTime=1452788988044, startRequestedTime=1452788988050, startedTime=1452788988050, finishTime=1452788988243, timeTaken=193, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=8155, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=25, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=462422016, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788988207, Process_Duration=8, Process_End_Time=1452788988215, Read_Duration=6, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1452788988068, firstTasksToStart=[ task_1452788987794_0001_1_01_000000 ], lastTaskFinishTime=1452788988239, lastTasksToFinish=[ task_1452788987794_0001_1_01_000000 ], minTaskDuration=171, maxTaskDuration=171, avgTaskDuration=171.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788987794_0001_1_01_000000 ], longestDurationTasks=[ task_1452788987794_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:48,252 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452788987794_0001_00_000002, Exiting
2016-01-14 16:29:48,252 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788987794_0001_00_000002
2016-01-14 16:29:48,252 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788987794_0001_00_000002
2016-01-14 16:29:48,252 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788987794_0001_1_01 [C656C04D1FD84351A746A11A4B11CD6E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:48,253 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788987794_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 16:29:48,253 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452788987794_0001_00_000002
2016-01-14 16:29:48,253 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452788987794_0001_00_000002 completed successfully
2016-01-14 16:29:48,253 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452788987794_0001_1_02 [F2A2F6D6AF4347EB95C68B0EB54535A0] attempt: attempt_1452788987794_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 16:29:48,254 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F2A2F6D6AF4347EB95C68B0EB54535A0
2016-01-14 16:29:48,254 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788987794_0001_1_02 [F2A2F6D6AF4347EB95C68B0EB54535A0]
2016-01-14 16:29:48,255 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788987794_0001_1_01 [C656C04D1FD84351A746A11A4B11CD6E] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 16:29:48,255 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788987794_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 16:29:48,256 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788987794_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 16:29:48,257 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:48,257 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:TASK_STARTED]: vertexName=F2A2F6D6AF4347EB95C68B0EB54535A0, taskId=task_1452788987794_0001_1_02_000000, scheduledTime=1452788988257, launchTime=1452788988257
2016-01-14 16:29:48,257 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788987794_0001_1_02_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:48,258 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788987794_0001_1_02 [F2A2F6D6AF4347EB95C68B0EB54535A0]
2016-01-14 16:29:48,258 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1B3AC423F00B45168A717F7CD97E3F89/D54D3F0F6EFF4E51B34B622B3C6DA7B2] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: F2A2F6D6AF4347EB95C68B0EB54535A0, VertexParallelism: 1, TaskAttemptID:attempt_1452788987794_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C134198274F146E7A28244CFC6E2A96F, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=C656C04D1FD84351A746A11A4B11CD6E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=96E1F871FA5243C9861E599B4477FA65, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:48,258 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788987794_0001_1_02_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:48,258 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:48,259 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788987794_0001_00_000003
2016-01-14 16:29:48,259 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788987794_0001_00_000003 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:48,261 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788987794_0001_00_000003
2016-01-14 16:29:48,262 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 16:29:48,264 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788987794_0001_00_000003 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:48,264 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:48,265 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788987794_0001_00_000003, launchTime=1452788988264
2016-01-14 16:29:48,265 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788987794_0001_00_000003
2016-01-14 16:29:48,266 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788987794_0001_1_02_000000_0] to container: [container_1452788987794_0001_00_000003]
2016-01-14 16:29:48,266 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788987794_0001_00_000003 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:48,266 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788987794_0001_00_000003 given task: attempt_1452788987794_0001_1_02_000000_0
2016-01-14 16:29:48,267 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788987794_0001_1_02_000000_0] started. Is using containerId: [container_1452788987794_0001_00_000003] on NM: [127.0.0.1:0]
2016-01-14 16:29:48,267 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F2A2F6D6AF4347EB95C68B0EB54535A0, taskAttemptId=attempt_1452788987794_0001_1_02_000000_0, startTime=1452788988266, containerId=container_1452788987794_0001_00_000003, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788987794_0001_00_000003/teamcity, completedLogs=
2016-01-14 16:29:48,267 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788987794_0001_1_02_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:48,267 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788987794_0001_1_02_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:48,267 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788987794_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788987794_0001_1_02_000000_0
2016-01-14 16:29:48,267 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:48 Starting to run new task attempt: attempt_1452788987794_0001_1_02_000000_0
2016-01-14 16:29:48,269 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788987794_0001_1_02_000000_0
2016-01-14 16:29:48,269 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:48,269 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:48,269 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:48,269 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:48,270 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:48,270 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:48,270 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1B3AC423F00B45168A717F7CD97E3F89/D54D3F0F6EFF4E51B34B622B3C6DA7B2] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: F2A2F6D6AF4347EB95C68B0EB54535A0, VertexParallelism: 1, TaskAttemptID:attempt_1452788987794_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C134198274F146E7A28244CFC6E2A96F, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=C656C04D1FD84351A746A11A4B11CD6E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=96E1F871FA5243C9861E599B4477FA65, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:48,270 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:48,271 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788987794_0001_1_02_000000_0
2016-01-14 16:29:48,271 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:48,273 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:48,272 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788987794_0001_1_02_000000_0 sent events: (0-2)
2016-01-14 16:29:48,274 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C134198274F146E7A28244CFC6E2A96F, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 16:29:48,275 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:48,275 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C134198274F146E7A28244CFC6E2A96F
2016-01-14 16:29:48,277 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 16:29:48,277 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 16:29:48,278 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C134198274F146E7A28244CFC6E2A96F
2016-01-14 16:29:48,278 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C134198274F146E7A28244CFC6E2A96F
2016-01-14 16:29:48,279 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C656C04D1FD84351A746A11A4B11CD6E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 16:29:48,279 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=96E1F871FA5243C9861E599B4477FA65, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 16:29:48,279 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:48,281 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:48,281 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C656C04D1FD84351A746A11A4B11CD6E
2016-01-14 16:29:48,306 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:48,306 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 96E1F871FA5243C9861E599B4477FA65
2016-01-14 16:29:48,306 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 16:29:48,306 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 96E1F871FA5243C9861E599B4477FA65
2016-01-14 16:29:48,306 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:48,307 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 16:29:48,311 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 16:29:48,311 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C656C04D1FD84351A746A11A4B11CD6E
2016-01-14 16:29:48,311 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C656C04D1FD84351A746A11A4B11CD6E
2016-01-14 16:29:48,317 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F2A2F6D6AF4347EB95C68B0EB54535A0, ordinal: 2
2016-01-14 16:29:48,317 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F2A2F6D6AF4347EB95C68B0EB54535A0, mem on start (mb), free: 238, total: 437, max: 672
2016-01-14 16:29:48,317 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:48,317 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 3 initializers to finish
2016-01-14 16:29:48,318 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:48,318 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:48,319 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 16:29:48,319 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 96E1F871FA5243C9861E599B4477FA65
2016-01-14 16:29:48,319 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:48,319 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:48,320 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 16:29:48,320 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 16:29:48,320 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:48,320 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 16:29:48,320 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345
2016-01-14 16:29:48,321 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 1268357504, TotalRequestedScaled: 6.0881160192E8, TotalJVMHeap: 704643072, TotalAvailable: 461541212, TotalRequested/TotalJVMHeap:1.80
2016-01-14 16:29:48,321 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C134198274F146E7A28244CFC6E2A96F, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-01-14 16:29:48,321 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 96E1F871FA5243C9861E599B4477FA65, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 16:29:48,321 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C656C04D1FD84351A746A11A4B11CD6E, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-01-14 16:29:48,321 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:48,322 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C134198274F146E7A28244CFC6E2A96F being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:48,322 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C134198274F146E7A28244CFC6E2A96F
2016-01-14 16:29:48,322 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C656C04D1FD84351A746A11A4B11CD6E
2016-01-14 16:29:48,322 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C656C04D1FD84351A746A11A4B11CD6E being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:48,323 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 16:29:48,323 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 16:29:48,323 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 16:29:48,323 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 16:29:48,324 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C656C04D1FD84351A746A11A4B11CD6E with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 16:29:48,324 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-01-14 16:29:48,324 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 16:29:48,324 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C134198274F146E7A28244CFC6E2A96F with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 16:29:48,325 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 16:29:48,325 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-01-14 16:29:48,325 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 16:29:48,326 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 2
2016-01-14 16:29:48,326 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 2 IOs to start
2016-01-14 16:29:48,326 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C656C04D1FD84351A746A11A4B11CD6E
2016-01-14 16:29:48,326 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 16:29:48,326 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:48,330 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C134198274F146E7A28244CFC6E2A96F
2016-01-14 16:29:48,330 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:48,331 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788987794_0001_1_02_000000_0
2016-01-14 16:29:48,331 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452788987794_0001_1_00_000000_0_10024, runDuration: 102663, ]
2016-01-14 16:29:48,332 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452788987794_0001_1_01_000000_0_10027, runDuration: 115028, ]
2016-01-14 16:29:48,333 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452788987794_0001_1_01_000000_0_10027] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1452788988333, TimeTaken=1, Rate=0.09 MB/s
2016-01-14 16:29:48,333 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C656C04D1FD84351A746A11A4B11CD6E
2016-01-14 16:29:48,333 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 16:29:48,334 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [C656C04D1FD84351A746A11A4B11CD6E] #4 in 2ms
2016-01-14 16:29:48,335 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452788987794_0001_1_00_000000_0_10024] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1452788988334, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 16:29:48,335 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C134198274F146E7A28244CFC6E2A96F
2016-01-14 16:29:48,335 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 16:29:48,335 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [C134198274F146E7A28244CFC6E2A96F] #5 in 1ms
2016-01-14 16:29:48,336 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]]
2016-01-14 16:29:48,338 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C656C04D1FD84351A746A11A4B11CD6E] #4, status:false, isInterrupted:false
2016-01-14 16:29:48,339 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 16:29:48,339 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-01-14 16:29:48,339 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] streamed: true, id: 0C2AECB450894ED3BFCC943F94A258B7
2016-01-14 16:29:48,340 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown"], id: 96E1F871FA5243C9861E599B4477FA65
2016-01-14 16:29:48,339 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 16:29:48,340 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 16:29:48,340 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C134198274F146E7A28244CFC6E2A96F] #5, status:false, isInterrupted:false
2016-01-14 16:29:48,340 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-01-14 16:29:48,341 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 16:29:48,340 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown"]
2016-01-14 16:29:48,341 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-01-14 16:29:48,341 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C656C04D1FD84351A746A11A4B11CD6E
2016-01-14 16:29:48,341 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 16:29:48,341 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 16:29:48,342 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-01-14 16:29:48,342 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C134198274F146E7A28244CFC6E2A96F
2016-01-14 16:29:48,342 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 16:29:48,341 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] 0C2AECB450894ED3BFCC943F94A258B7, for 2 inputs
2016-01-14 16:29:48,341 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 16:29:48,344 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] 0C2AECB450894ED3BFCC943F94A258B7, for 2 inputs
2016-01-14 16:29:48,350 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(89)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2016-01-14 16:29:48,350 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(94)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2016-01-14 16:29:48,355 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F2A2F6D6AF4347EB95C68B0EB54535A0, all 2 inputs ready in: 00:00:00.001
2016-01-14 16:29:48,355 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@22157e6b; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@311eb448
2016-01-14 16:29:48,356 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@56cdfc29; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@6d661444
2016-01-14 16:29:48,359 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 16:29:48,360 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 16:29:48,360 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452788987794_0001_1_02_000000_0
2016-01-14 16:29:48,361 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452788987794_0001_1_02_000000_0 given a go for committing the task output.
2016-01-14 16:29:48,361 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788987794_0001_1_02_000000_0
2016-01-14 16:29:48,361 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F2A2F6D6AF4347EB95C68B0EB54535A0, mem on close (mb), free: 233, total: 437, max: 672
2016-01-14 16:29:48,362 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C134198274F146E7A28244CFC6E2A96F
2016-01-14 16:29:48,362 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 16:29:48,362 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 16:29:48,362 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 16:29:48,362 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 16:29:48,363 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 16:29:48,363 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C656C04D1FD84351A746A11A4B11CD6E
2016-01-14 16:29:48,363 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 16:29:48,363 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 16:29:48,364 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 16:29:48,364 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 16:29:48,364 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 16:29:48 Completed running task attempt: attempt_1452788987794_0001_1_02_000000_0
2016-01-14 16:29:48,365 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788987794_0001_1_02_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:48,366 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=20, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=458227712, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=18, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=11][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452788988355, Process_Duration=6, Process_End_Time=1452788988361, Read_Duration=0, Tuples_Read=10, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-01-14 16:29:48,367 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:48,367 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:48,367 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:48,367 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F2A2F6D6AF4347EB95C68B0EB54535A0, taskAttemptId=attempt_1452788987794_0001_1_02_000000_0, startTime=1452788988266, finishTime=1452788988366, timeTaken=100, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=20, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=458227712, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=18, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=11, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788988355, Process_Duration=6, Process_End_Time=1452788988361, Read_Duration=0, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-01-14 16:29:48,367 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788987794_0001_00_000003
2016-01-14 16:29:48,367 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788987794_0001_1_02_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:48,368 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:48,368 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788987794_0001_1_02_000000_0
2016-01-14 16:29:48,369 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:48,369 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:TASK_FINISHED]: vertexName=F2A2F6D6AF4347EB95C68B0EB54535A0, taskId=task_1452788987794_0001_1_02_000000, startTime=1452788988266, finishTime=1452788988368, timeTaken=102, status=SUCCEEDED, successfulAttemptID=attempt_1452788987794_0001_1_02_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=20, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=458227712, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=18, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=11, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788988355, Process_Duration=6, Process_End_Time=1452788988361, Read_Duration=0, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-01-14 16:29:48,370 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788987794_0001_1_02_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:48,370 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788987794_0001_1_02 [F2A2F6D6AF4347EB95C68B0EB54535A0] : 1
2016-01-14 16:29:48,370 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788987794_0001_1_02 [F2A2F6D6AF4347EB95C68B0EB54535A0]
2016-01-14 16:29:48,371 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:48,372 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:VERTEX_FINISHED]: vertexName=F2A2F6D6AF4347EB95C68B0EB54535A0, vertexId=vertex_1452788987794_0001_1_02, initRequestedTime=1452788988001, initedTime=1452788988020, startRequestedTime=1452788988055, startedTime=1452788988055, finishTime=1452788988370, timeTaken=315, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=20, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=458227712, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=18, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=11, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788988355, Process_Duration=6, Process_End_Time=1452788988361, Read_Duration=0, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1452788988266, firstTasksToStart=[ task_1452788987794_0001_1_02_000000 ], lastTaskFinishTime=1452788988366, lastTasksToFinish=[ task_1452788987794_0001_1_02_000000 ], minTaskDuration=100, maxTaskDuration=100, avgTaskDuration=100.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788987794_0001_1_02_000000 ], longestDurationTasks=[ task_1452788987794_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:48,372 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788987794_0001_1_02 [F2A2F6D6AF4347EB95C68B0EB54535A0] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:48,372 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788987794_0001_00_000003 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:48,373 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:48,373 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788987794_0001_00_000003, stoppedTime=1452788988372, exitStatus=0
2016-01-14 16:29:48,373 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788987794_0001_00_000003 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:48,374 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788987794_0001_1_02 [F2A2F6D6AF4347EB95C68B0EB54535A0] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 16:29:48,374 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452788987794_0001_1
2016-01-14 16:29:48,374 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452788987794_0001_1
2016-01-14 16:29:48,375 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452788987794_0001_1_00 [C134198274F146E7A28244CFC6E2A96F]
2016-01-14 16:29:48,375 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452788987794_0001_1_01 [C656C04D1FD84351A746A11A4B11CD6E]
2016-01-14 16:29:48,375 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 96E1F871FA5243C9861E599B4477FA65 for vertex: vertex_1452788987794_0001_1_02 [F2A2F6D6AF4347EB95C68B0EB54535A0]
2016-01-14 16:29:48,380 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452788987794_0001_00_000003 is valid, but no longer registered, and will be killed
2016-01-14 16:29:48,380 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788987794_0001_00_000003: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 16:29:48,380 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452788987794_0001_00_000003, Exiting
2016-01-14 16:29:48,380 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788987794_0001_00_000003
2016-01-14 16:29:48,381 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788987794_0001_00_000003
2016-01-14 16:29:48,383 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788987794_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 16:29:48,384 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452788987794_0001_00_000003 completed successfully
2016-01-14 16:29:48,391 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452788987794_0001_1, queueSize=0
2016-01-14 16:29:48,391 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 16:29:48,393 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788987794_0001_1][Event:DAG_FINISHED]: dagId=dag_1452788987794_0001_1, startTime=1452788987996, finishTime=1452788988388, timeTaken=392, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=2, File System Counters, FILE_BYTES_READ=440, FILE_BYTES_WRITTEN=8334, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=70, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1383071744, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=15, OUTPUT_BYTES=150, OUTPUT_BYTES_WITH_OVERHEAD=182, OUTPUT_BYTES_PHYSICAL=190, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=18, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=11, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4358366964744, Process_Duration=19, Process_End_Time=4358366964763, Read_Duration=8, Tuples_Read=20, Tuples_Written=15, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=5
2016-01-14 16:29:48,393 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452788987794_0001_1 finished with state: SUCCEEDED
2016-01-14 16:29:48,393 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788987794_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 16:29:48,394 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452788987794_0001_00_000003
2016-01-14 16:29:48,394 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788987794_0001_00_000003 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 16:29:48 Completed Dag: dag_1452788987794_0001_1
2016-01-14 16:29:48,395 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452788987794_0001_1, dagState=SUCCEEDED
2016-01-14 16:29:48,395 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788987794_0001_1_post
2016-01-14 16:29:48,395 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:48,396 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 16:29:48,404 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lexcogrouprelativeunknown, applicationId=application_1452788987794_0001
2016-01-14 16:29:48,406 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 16:29:48,406 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 16:29:48,407 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 16:29:48,407 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 16:29:48,407 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 16:29:48,408 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 16:29:48,408 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 16:29:48,409 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 16:29:48,409 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 16:29:48,409 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 16:29:48,410 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 16:29:48,410 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 16:29:48,411 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 16:29:48,416 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1B3AC423F00B45168A717F7CD97E3F89/D54D3F0F6EFF4E51B34B622B3C6DA7B2/.tez/application_1452788987794_0001
2016-01-14 16:29:48,416 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] completed in: 00:00.655
2016-01-14 16:29:48,490 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown
2016-01-14 16:29:48,515 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 16:29:48,515 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 16:29:48,562 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.021
2016-01-14 16:29:48,583 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.042
2016-01-14 16:29:48,584 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 16:29:48,584 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 16:29:48,586 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 16:29:48,607 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-01-14 16:29:48,621 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative
2016-01-14 16:29:48,675 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 16:29:48,675 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 16:29:48,675 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative"]
2016-01-14 16:29:48,675 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 16:29:48,676 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 16:29:48,676 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 16:29:48,676 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...leRelative/simplerelative
2016-01-14 16:29:48,677 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 16:29:48,690 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 16:29:48,695 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:48,696 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 16:29:48,696 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 16:29:48,705 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/F475903C5431465B8832A106F38A02B6/CC3D0CDACE4D4DE2B84BCD1B6C623BBF/.tez/application_1452788988696_0001 doesn't exist and is created
2016-01-14 16:29:48,722 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 16:29:48,722 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 16:29:48,722 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/F475903C5431465B8832A106F38A02B6/CC3D0CDACE4D4DE2B84BCD1B6C623BBF/.tez/application_1452788988696_0001
2016-01-14 16:29:48,726 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452788988696_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:48,737 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 16:29:48,738 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 16:29:48,740 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 16:29:48,740 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 16:29:48,740 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/F475903C5431465B8832A106F38A02B6/CC3D0CDACE4D4DE2B84BCD1B6C623BBF/.tez/application_1452788988696_0001/localmode-log-dir
2016-01-14 16:29:48,741 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/F475903C5431465B8832A106F38A02B6/CC3D0CDACE4D4DE2B84BCD1B6C623BBF/.tez/application_1452788988696_0001/localmode-log-dir/history.txt.appattempt_1452788988696_0001_000000, maxErrors=10
2016-01-14 16:29:48,741 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 16:29:48,741 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452788988696_0001_000000, appSubmitTime=1452788988726, launchTime=1452788988726
2016-01-14 16:29:48,745 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 16:29:48,746 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 16:29:48,747 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 16:29:48,748 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 16:29:48,752 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:44637
2016-01-14 16:29:48,753 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 16:29:48,755 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452788988696_0001_000000, startTime=1452788988755
2016-01-14 16:29:48,755 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 16:29:48,755 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 16:29:48,822 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 16:29:48,822 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 16:29:48,822 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...leRelative/simplerelative, applicationId=application_1452788988696_0001, dagName=[F475903C5431465B8832A106F38A02B6/CC3D0CDACE4D4DE2B84BCD1B6C623BBF] test/(1/1) ...leRelative/simplerelative
2016-01-14 16:29:48,842 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [F475903C5431465B8832A106F38A02B6/CC3D0CDACE4D4DE2B84BCD1B6C623BBF] test/(1/1) ...leRelative/simplerelative
2016-01-14 16:29:48,844 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452788988696_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/F475903C5431465B8832A106F38A02B6/CC3D0CDACE4D4DE2B84BCD1B6C623BBF/.tez/application_1452788988696_0001/localmode-log-dir/dag_1452788988696_0001_1.dot
2016-01-14 16:29:48,845 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/F475903C5431465B8832A106F38A02B6/CC3D0CDACE4D4DE2B84BCD1B6C623BBF/.tez/application_1452788988696_0001/localmode-log-dir/dag_1452788988696_0001_1-tez-dag.pb.txt
2016-01-14 16:29:48,868 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788988696_0001_1
2016-01-14 16:29:48,868 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:48,868 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [F475903C5431465B8832A106F38A02B6/CC3D0CDACE4D4DE2B84BCD1B6C623BBF] test/(1/1) ...leRelative/simplerelative
2016-01-14 16:29:48 Running Dag: dag_1452788988696_0001_1
2016-01-14 16:29:48,892 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 16:29:48,893 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788988696_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452788988696_0001_1, submitTime=1452788988843
2016-01-14 16:29:48,894 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FE6F7F9372B94E1B832360F2C2A761E7
2016-01-14 16:29:48,894 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 16:29:48,895 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 16:29:48,895 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788988696_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452788988696_0001_1, initTime=1452788988893
2016-01-14 16:29:48,895 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788988696_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 16:29:48,897 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788988696_0001_1][Event:DAG_STARTED]: dagID=dag_1452788988696_0001_1, startTime=1452788988896
2016-01-14 16:29:48,897 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 16:29:48,897 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788988696_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 16:29:48,897 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B07960E9FDFF4523B21ED96B40BAC6C0 : {86C73F50B86141CEBF2B76FE4F1E2E77={InputName=86C73F50B86141CEBF2B76FE4F1E2E77}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 16:29:48,897 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 86C73F50B86141CEBF2B76FE4F1E2E77, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 16:29:48,898 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452788988696_0001_1_00 [B07960E9FDFF4523B21ED96B40BAC6C0]
2016-01-14 16:29:48,898 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452788988696_0001_1_00 [B07960E9FDFF4523B21ED96B40BAC6C0]
2016-01-14 16:29:48,898 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452788988696_0001_1_00 [B07960E9FDFF4523B21ED96B40BAC6C0]
2016-01-14 16:29:48,898 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452788988696_0001_1_00 [B07960E9FDFF4523B21ED96B40BAC6C0]. Starting root input initializers: 1
2016-01-14 16:29:48,899 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 16:29:48,899 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788988696_0001_1_00 [B07960E9FDFF4523B21ED96B40BAC6C0] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 16:29:48,899 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452788988696_0001_1_01 [FE6F7F9372B94E1B832360F2C2A761E7]
2016-01-14 16:29:48,902 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...leRelative/simplerelative, applicationId=application_1452788988696_0001, dagName=[F475903C5431465B8832A106F38A02B6/CC3D0CDACE4D4DE2B84BCD1B6C623BBF] test/(1/1) ...leRelative/simplerelative
2016-01-14 16:29:48,903 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 16:29:48,903 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452788988696_0001_1_01 [FE6F7F9372B94E1B832360F2C2A761E7]
2016-01-14 16:29:48,903 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452788988696_0001_1_01 [FE6F7F9372B94E1B832360F2C2A761E7]
2016-01-14 16:29:48,904 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452788988696_0001_1_01 [FE6F7F9372B94E1B832360F2C2A761E7]
2016-01-14 16:29:48,904 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=E086FDBFFE384C8F9A438BC227036D33, vertexId=vertex_1452788988696_0001_1_01 [FE6F7F9372B94E1B832360F2C2A761E7], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 16:29:48,904 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=E086FDBFFE384C8F9A438BC227036D33, vertexId=vertex_1452788988696_0001_1_01 [FE6F7F9372B94E1B832360F2C2A761E7]
2016-01-14 16:29:48,907 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=E086FDBFFE384C8F9A438BC227036D33, vertexName=FE6F7F9372B94E1B832360F2C2A761E7, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 16:29:48,908 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=E086FDBFFE384C8F9A438BC227036D33, vertexId=vertex_1452788988696_0001_1_01 [FE6F7F9372B94E1B832360F2C2A761E7]
2016-01-14 16:29:48,910 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:48,910 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788988696_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FE6F7F9372B94E1B832360F2C2A761E7, vertexId=vertex_1452788988696_0001_1_01, initRequestedTime=1452788988899, initedTime=1452788988910, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 16:29:48,910 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788988696_0001_1_01 [FE6F7F9372B94E1B832360F2C2A761E7] transitioned from NEW to INITED due to event V_INIT
2016-01-14 16:29:48,917 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 86C73F50B86141CEBF2B76FE4F1E2E77 on vertex vertex_1452788988696_0001_1_00 [B07960E9FDFF4523B21ED96B40BAC6C0]
2016-01-14 16:29:48,920 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 16:29:48,921 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 86C73F50B86141CEBF2B76FE4F1E2E77 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 16:29:48,922 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 16:29:48,922 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 16:29:48,927 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-01-14 16:29:48,928 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 86C73F50B86141CEBF2B76FE4F1E2E77 on vertex vertex_1452788988696_0001_1_00 [B07960E9FDFF4523B21ED96B40BAC6C0]
2016-01-14 16:29:48,928 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788988696_0001_1_00 [B07960E9FDFF4523B21ED96B40BAC6C0] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:48,929 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {86C73F50B86141CEBF2B76FE4F1E2E77=forAllWorkUnits=true, update=[1]}
2016-01-14 16:29:48,929 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452788988696_0001_1_00 [B07960E9FDFF4523B21ED96B40BAC6C0] parallelism set to 1
2016-01-14 16:29:48,929 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788988696_0001_1_00 [B07960E9FDFF4523B21ED96B40BAC6C0] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:48,930 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788988696_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788988696_0001_1_00, eventCount=1
2016-01-14 16:29:48,931 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:48,931 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788988696_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B07960E9FDFF4523B21ED96B40BAC6C0, vertexId=vertex_1452788988696_0001_1_00, initRequestedTime=1452788988897, initedTime=1452788988930, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 16:29:48,932 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788988696_0001_1_00 [B07960E9FDFF4523B21ED96B40BAC6C0] with distanceFromRoot: 0
2016-01-14 16:29:48,932 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788988696_0001_1_00 [B07960E9FDFF4523B21ED96B40BAC6C0] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 16:29:48,932 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B07960E9FDFF4523B21ED96B40BAC6C0
2016-01-14 16:29:48,932 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788988696_0001_1_00 [B07960E9FDFF4523B21ED96B40BAC6C0]
2016-01-14 16:29:48,933 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:48,933 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788988696_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788988696_0001_1_00, startRequestedTime=1452788988932, startedTime=1452788988932
2016-01-14 16:29:48,933 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788988696_0001_1_00 [B07960E9FDFF4523B21ED96B40BAC6C0] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:48,934 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:48,934 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788988696_0001_1][Event:TASK_STARTED]: vertexName=B07960E9FDFF4523B21ED96B40BAC6C0, taskId=task_1452788988696_0001_1_00_000000, scheduledTime=1452788988934, launchTime=1452788988934
2016-01-14 16:29:48,935 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788988696_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:48,935 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452788988696_0001_1_00 for vertex: vertex_1452788988696_0001_1_01 [FE6F7F9372B94E1B832360F2C2A761E7] numStartedSources: 1 numSources: 1
2016-01-14 16:29:48,935 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788988696_0001_1_01 [FE6F7F9372B94E1B832360F2C2A761E7] with distanceFromRoot: 1
2016-01-14 16:29:48,936 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788988696_0001_1_00 [B07960E9FDFF4523B21ED96B40BAC6C0]
2016-01-14 16:29:48,936 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B07960E9FDFF4523B21ED96B40BAC6C0 in vertex: FE6F7F9372B94E1B832360F2C2A761E7
2016-01-14 16:29:48,936 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: FE6F7F9372B94E1B832360F2C2A761E7 with 1 source tasks and 1 pending tasks
2016-01-14 16:29:48,937 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:48,937 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788988696_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788988696_0001_1_01, startRequestedTime=1452788988936, startedTime=1452788988936
2016-01-14 16:29:48,938 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788988696_0001_1_01 [FE6F7F9372B94E1B832360F2C2A761E7] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:48,938 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [F475903C5431465B8832A106F38A02B6/CC3D0CDACE4D4DE2B84BCD1B6C623BBF] test/(1/1) ...leRelative/simplerelative, VertexName: B07960E9FDFF4523B21ED96B40BAC6C0, VertexParallelism: 1, TaskAttemptID:attempt_1452788988696_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=86C73F50B86141CEBF2B76FE4F1E2E77, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FE6F7F9372B94E1B832360F2C2A761E7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 16:29:48,938 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788988696_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:48,939 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:48,943 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452788988696_0001, with dag id: dag_1452788988696_0001_1
2016-01-14 16:29:48,945 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 16:29:48,945 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 16:29:48,946 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 16:29:48,953 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788988696_0001_00_000001
2016-01-14 16:29:48,954 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788988696_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:48,956 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788988696_0001_00_000001
2016-01-14 16:29:48,956 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 16:29:48,959 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788988696_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:48,959 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:48,960 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788988696_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788988696_0001_00_000001, launchTime=1452788988959
2016-01-14 16:29:48,961 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788988696_0001_00_000001
2016-01-14 16:29:48,962 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788988696_0001_1_00_000000_0] to container: [container_1452788988696_0001_00_000001]
2016-01-14 16:29:48,962 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788988696_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:48,963 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788988696_0001_00_000001 given task: attempt_1452788988696_0001_1_00_000000_0
2016-01-14 16:29:48,963 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788988696_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788988696_0001_1_00_000000_0
2016-01-14 16:29:48,963 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788988696_0001_1_00_000000_0] started. Is using containerId: [container_1452788988696_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 16:29:48,964 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788988696_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B07960E9FDFF4523B21ED96B40BAC6C0, taskAttemptId=attempt_1452788988696_0001_1_00_000000_0, startTime=1452788988963, containerId=container_1452788988696_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788988696_0001_00_000001/teamcity, completedLogs=
2016-01-14 16:29:48,964 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:48,964 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788988696_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:48,964 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788988696_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:48 Starting to run new task attempt: attempt_1452788988696_0001_1_00_000000_0
2016-01-14 16:29:48,964 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788988696_0001_1_00_000000_0
2016-01-14 16:29:48,965 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:48,965 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:48,965 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:48,965 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:48,965 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:48,965 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:48,966 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [F475903C5431465B8832A106F38A02B6/CC3D0CDACE4D4DE2B84BCD1B6C623BBF] test/(1/1) ...leRelative/simplerelative, VertexName: B07960E9FDFF4523B21ED96B40BAC6C0, VertexParallelism: 1, TaskAttemptID:attempt_1452788988696_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=86C73F50B86141CEBF2B76FE4F1E2E77, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FE6F7F9372B94E1B832360F2C2A761E7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 16:29:48,966 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:48,966 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788988696_0001_1_00_000000_0
2016-01-14 16:29:48,966 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:48,967 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:48,970 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=86C73F50B86141CEBF2B76FE4F1E2E77, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 16:29:48,970 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:48,971 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 86C73F50B86141CEBF2B76FE4F1E2E77
2016-01-14 16:29:48,971 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 86C73F50B86141CEBF2B76FE4F1E2E77
2016-01-14 16:29:48,974 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:48,977 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:48,981 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FE6F7F9372B94E1B832360F2C2A761E7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 16:29:48,982 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:48,983 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FE6F7F9372B94E1B832360F2C2A761E7
2016-01-14 16:29:48,984 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788988696_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 16:29:48,986 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 16:29:48,987 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 86C73F50B86141CEBF2B76FE4F1E2E77
2016-01-14 16:29:48,987 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 86C73F50B86141CEBF2B76FE4F1E2E77
2016-01-14 16:29:48,991 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B07960E9FDFF4523B21ED96B40BAC6C0, ordinal: 0
2016-01-14 16:29:48,991 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B07960E9FDFF4523B21ED96B40BAC6C0, mem on start (mb), free: 139, total: 437, max: 672
2016-01-14 16:29:48,992 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:48,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:48,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:48,994 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 16:29:48,994 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: FE6F7F9372B94E1B832360F2C2A761E7
2016-01-14 16:29:48,995 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FE6F7F9372B94E1B832360F2C2A761E7
2016-01-14 16:29:48,996 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:48,996 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:48,997 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 16:29:48,997 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:48,997 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 16:29:48,997 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 16:29:48,997 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 16:29:48,997 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 86C73F50B86141CEBF2B76FE4F1E2E77, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 16:29:48,998 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FE6F7F9372B94E1B832360F2C2A761E7, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 16:29:48,998 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:49,000 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 86C73F50B86141CEBF2B76FE4F1E2E77
2016-01-14 16:29:49,000 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:49,000 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 86C73F50B86141CEBF2B76FE4F1E2E77
2016-01-14 16:29:49,001 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 86C73F50B86141CEBF2B76FE4F1E2E77 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:49,001 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 16:29:49,002 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:49,002 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:49,003 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788988696_0001_1_00_000000_0
2016-01-14 16:29:49,005 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 16:29:49,006 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 16:29:49,007 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 86C73F50B86141CEBF2B76FE4F1E2E77
2016-01-14 16:29:49,008 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: A9BF34ECBDE546B4AA713FAEDDD5A195
2016-01-14 16:29:49,008 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@64948189
2016-01-14 16:29:49,008 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] A9BF34ECBDE546B4AA713FAEDDD5A195
2016-01-14 16:29:49,009 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@446e3aba; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@20d1ac6f; comparator=cascading.tuple.hadoop.util.TupleComparator@258fbc3e; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 16:29:49,011 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 16:29:49,011 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 16:29:49,012 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 16:29:49,013 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@64948189
2016-01-14 16:29:49,014 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 16:29:49,014 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 16:29:49,026 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 16:29:49,026 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 16:29:49,027 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 16:29:49,027 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 16:29:49,027 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 16:29:49,028 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 16:29:49,028 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:49,029 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B07960E9FDFF4523B21ED96B40BAC6C0, all 1 inputs ready in: 00:00:00.000
2016-01-14 16:29:49,032 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788988696_0001_1_00_000000_0
2016-01-14 16:29:49,032 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B07960E9FDFF4523B21ED96B40BAC6C0, mem on close (mb), free: 36, total: 437, max: 672
2016-01-14 16:29:49,032 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 16:29:49,032 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 16:29:49,032 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 210; bufvoid = 104857600
2016-01-14 16:29:49,032 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214360(104857440); length = 37/6553600
2016-01-14 16:29:49,036 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 16:29:49 Completed running task attempt: attempt_1452788988696_0001_1_00_000000_0
2016-01-14 16:29:49,042 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788988696_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:49,043 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788988696_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788988696_0001_1_00, eventCount=1
2016-01-14 16:29:49,043 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=458227712, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452788989029, Process_Duration=3, Process_End_Time=1452788989032, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10]]
2016-01-14 16:29:49,044 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:49,044 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:49,044 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788988696_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B07960E9FDFF4523B21ED96B40BAC6C0, taskAttemptId=attempt_1452788988696_0001_1_00_000000_0, startTime=1452788988963, finishTime=1452788989044, timeTaken=81, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=458227712, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788989029, Process_Duration=3, Process_End_Time=1452788989032, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-01-14 16:29:49,044 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:49,045 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788988696_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:49,045 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788988696_0001_00_000001
2016-01-14 16:29:49,045 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788988696_0001_1_00_000000_0
2016-01-14 16:29:49,046 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:49,047 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:49,047 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788988696_0001_1][Event:TASK_FINISHED]: vertexName=B07960E9FDFF4523B21ED96B40BAC6C0, taskId=task_1452788988696_0001_1_00_000000, startTime=1452788988963, finishTime=1452788989046, timeTaken=83, status=SUCCEEDED, successfulAttemptID=attempt_1452788988696_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=458227712, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788989029, Process_Duration=3, Process_End_Time=1452788989032, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-01-14 16:29:49,047 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788988696_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:49,047 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788988696_0001_1_00 [B07960E9FDFF4523B21ED96B40BAC6C0] : 1
2016-01-14 16:29:49,047 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788988696_0001_1_00 [B07960E9FDFF4523B21ED96B40BAC6C0]
2016-01-14 16:29:49,049 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:49,049 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788988696_0001_1][Event:VERTEX_FINISHED]: vertexName=B07960E9FDFF4523B21ED96B40BAC6C0, vertexId=vertex_1452788988696_0001_1_00, initRequestedTime=1452788988897, initedTime=1452788988930, startRequestedTime=1452788988932, startedTime=1452788988932, finishTime=1452788989048, timeTaken=116, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=458227712, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788989029, Process_Duration=3, Process_End_Time=1452788989032, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1452788988963, firstTasksToStart=[ task_1452788988696_0001_1_00_000000 ], lastTaskFinishTime=1452788989044, lastTasksToFinish=[ task_1452788988696_0001_1_00_000000 ], minTaskDuration=81, maxTaskDuration=81, avgTaskDuration=81.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788988696_0001_1_00_000000 ], longestDurationTasks=[ task_1452788988696_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:49,049 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788988696_0001_1_00 [B07960E9FDFF4523B21ED96B40BAC6C0] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:49,050 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788988696_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:49,050 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788988696_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788988696_0001_00_000001, stoppedTime=1452788989050, exitStatus=0
2016-01-14 16:29:49,051 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788988696_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:49,052 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452788988696_0001_1_01 [FE6F7F9372B94E1B832360F2C2A761E7] attempt: attempt_1452788988696_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 16:29:49,053 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: FE6F7F9372B94E1B832360F2C2A761E7
2016-01-14 16:29:49,053 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788988696_0001_1_01 [FE6F7F9372B94E1B832360F2C2A761E7]
2016-01-14 16:29:49,054 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788988696_0001_1_00 [B07960E9FDFF4523B21ED96B40BAC6C0] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 16:29:49,054 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452788988696_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 16:29:49,051 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:49,055 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788988696_0001_1][Event:TASK_STARTED]: vertexName=FE6F7F9372B94E1B832360F2C2A761E7, taskId=task_1452788988696_0001_1_01_000000, scheduledTime=1452788989055, launchTime=1452788989055
2016-01-14 16:29:49,056 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788988696_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:49,056 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:49,057 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452788988696_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 16:29:49,058 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788988696_0001_00_000001: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 16:29:49,057 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788988696_0001_1_01 [FE6F7F9372B94E1B832360F2C2A761E7]
2016-01-14 16:29:49,059 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788988696_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 16:29:49,060 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [F475903C5431465B8832A106F38A02B6/CC3D0CDACE4D4DE2B84BCD1B6C623BBF] test/(1/1) ...leRelative/simplerelative, VertexName: FE6F7F9372B94E1B832360F2C2A761E7, VertexParallelism: 1, TaskAttemptID:attempt_1452788988696_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B07960E9FDFF4523B21ED96B40BAC6C0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=E086FDBFFE384C8F9A438BC227036D33, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:49,060 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788988696_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:49,060 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452788988696_0001_00_000001, Exiting
2016-01-14 16:29:49,061 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788988696_0001_00_000001
2016-01-14 16:29:49,061 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788988696_0001_00_000001
2016-01-14 16:29:49,065 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:49,066 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788988696_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 16:29:49,067 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452788988696_0001_00_000001 completed successfully
2016-01-14 16:29:49,067 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788988696_0001_00_000002
2016-01-14 16:29:49,067 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788988696_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:49,068 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788988696_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 16:29:49,071 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788988696_0001_00_000002
2016-01-14 16:29:49,072 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 16:29:49,075 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788988696_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:49,075 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:49,076 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788988696_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788988696_0001_00_000002, launchTime=1452788989075
2016-01-14 16:29:49,077 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788988696_0001_00_000002
2016-01-14 16:29:49,078 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788988696_0001_1_01_000000_0] to container: [container_1452788988696_0001_00_000002]
2016-01-14 16:29:49,078 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788988696_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:49,078 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788988696_0001_00_000002 given task: attempt_1452788988696_0001_1_01_000000_0
2016-01-14 16:29:49,079 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788988696_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788988696_0001_1_01_000000_0
2016-01-14 16:29:49,079 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788988696_0001_1_01_000000_0] started. Is using containerId: [container_1452788988696_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 16:29:49,079 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788988696_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FE6F7F9372B94E1B832360F2C2A761E7, taskAttemptId=attempt_1452788988696_0001_1_01_000000_0, startTime=1452788989079, containerId=container_1452788988696_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788988696_0001_00_000002/teamcity, completedLogs=
2016-01-14 16:29:49,080 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788988696_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:49 Starting to run new task attempt: attempt_1452788988696_0001_1_01_000000_0
2016-01-14 16:29:49,080 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788988696_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:49,080 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:49,080 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788988696_0001_1_01_000000_0
2016-01-14 16:29:49,081 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:49,081 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:49,081 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:49,082 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:49,082 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:49,082 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:49,083 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [F475903C5431465B8832A106F38A02B6/CC3D0CDACE4D4DE2B84BCD1B6C623BBF] test/(1/1) ...leRelative/simplerelative, VertexName: FE6F7F9372B94E1B832360F2C2A761E7, VertexParallelism: 1, TaskAttemptID:attempt_1452788988696_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B07960E9FDFF4523B21ED96B40BAC6C0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=E086FDBFFE384C8F9A438BC227036D33, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:49,083 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:49,084 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788988696_0001_1_01_000000_0
2016-01-14 16:29:49,084 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:49,084 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:49,086 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B07960E9FDFF4523B21ED96B40BAC6C0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 16:29:49,087 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:49,087 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B07960E9FDFF4523B21ED96B40BAC6C0
2016-01-14 16:29:49,089 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 16:29:49,089 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 16:29:49,097 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E086FDBFFE384C8F9A438BC227036D33, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 16:29:49,098 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:49,098 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E086FDBFFE384C8F9A438BC227036D33
2016-01-14 16:29:49,098 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 16:29:49,098 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: E086FDBFFE384C8F9A438BC227036D33
2016-01-14 16:29:49,097 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788988696_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 16:29:49,090 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:49,097 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B07960E9FDFF4523B21ED96B40BAC6C0
2016-01-14 16:29:49,106 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:49,113 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 16:29:49,113 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E086FDBFFE384C8F9A438BC227036D33
2016-01-14 16:29:49,114 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B07960E9FDFF4523B21ED96B40BAC6C0
2016-01-14 16:29:49,116 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FE6F7F9372B94E1B832360F2C2A761E7, ordinal: 1
2016-01-14 16:29:49,117 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FE6F7F9372B94E1B832360F2C2A761E7, mem on start (mb), free: 31, total: 437, max: 672
2016-01-14 16:29:49,117 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:49,117 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:49,118 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:49,118 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:49,119 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:49,119 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 16:29:49,119 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:49,119 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 16:29:49,120 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 16:29:49,120 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 16:29:49,120 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E086FDBFFE384C8F9A438BC227036D33, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 16:29:49,121 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B07960E9FDFF4523B21ED96B40BAC6C0, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 16:29:49,121 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:49,121 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B07960E9FDFF4523B21ED96B40BAC6C0 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:49,121 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B07960E9FDFF4523B21ED96B40BAC6C0
2016-01-14 16:29:49,121 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 16:29:49,122 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:49,122 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 16:29:49,123 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 16:29:49,124 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B07960E9FDFF4523B21ED96B40BAC6C0 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 16:29:49,124 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 16:29:49,124 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 16:29:49,125 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 16:29:49,126 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B07960E9FDFF4523B21ED96B40BAC6C0
2016-01-14 16:29:49,127 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:49,128 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788988696_0001_1_01_000000_0
2016-01-14 16:29:49,378 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-01-14 16:29:49,379 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: A9BF34ECBDE546B4AA713FAEDDD5A195
2016-01-14 16:29:49,380 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative"], id: E086FDBFFE384C8F9A438BC227036D33
2016-01-14 16:29:49,381 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452788988696_0001_1_00_000000_0_10034, runDuration: 56197, ]
2016-01-14 16:29:49,381 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative"]
2016-01-14 16:29:49,381 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] A9BF34ECBDE546B4AA713FAEDDD5A195, for 1 inputs
2016-01-14 16:29:49,382 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452788988696_0001_1_00_000000_0_10034] to DISK_DIRECT, CompressedSize=204, DecompressedSize=200,EndTime=1452788989382, TimeTaken=1, Rate=0.19 MB/s
2016-01-14 16:29:49,382 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B07960E9FDFF4523B21ED96B40BAC6C0
2016-01-14 16:29:49,383 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 16:29:49,383 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [B07960E9FDFF4523B21ED96B40BAC6C0] #6 in 2ms
2016-01-14 16:29:49,383 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B07960E9FDFF4523B21ED96B40BAC6C0] #6, status:false, isInterrupted:false
2016-01-14 16:29:49,384 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 16:29:49,384 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 204 bytes from disk
2016-01-14 16:29:49,385 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 16:29:49,385 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 16:29:49,385 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 175 bytes
2016-01-14 16:29:49,385 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B07960E9FDFF4523B21ED96B40BAC6C0
2016-01-14 16:29:49,386 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 16:29:49,387 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FE6F7F9372B94E1B832360F2C2A761E7, all 1 inputs ready in: 00:00:00.006
2016-01-14 16:29:49,388 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@64f2303d; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@583d2891
2016-01-14 16:29:49,391 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 16:29:49,392 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 16:29:49,392 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452788988696_0001_1_01_000000_0
2016-01-14 16:29:49,393 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452788988696_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 16:29:49,395 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788988696_0001_1_01_000000_0
2016-01-14 16:29:49,395 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FE6F7F9372B94E1B832360F2C2A761E7, mem on close (mb), free: 336, total: 357, max: 672
2016-01-14 16:29:49,395 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B07960E9FDFF4523B21ED96B40BAC6C0
2016-01-14 16:29:49,396 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 16:29:49,397 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 16:29:49,397 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 16:29:49,398 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 16:29:49,398 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 16:29:49 Completed running task attempt: attempt_1452788988696_0001_1_01_000000_0
2016-01-14 16:29:49,399 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788988696_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:49,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=245, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=260, MERGE_PHASE_TIME=262, FIRST_EVENT_RECEIVED=258, LAST_EVENT_RECEIVED=258][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452788989387, Process_Duration=8, Process_End_Time=1452788989395, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=8]]
2016-01-14 16:29:49,401 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:49,401 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:49,401 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:49,402 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788988696_0001_00_000002
2016-01-14 16:29:49,401 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788988696_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FE6F7F9372B94E1B832360F2C2A761E7, taskAttemptId=attempt_1452788988696_0001_1_01_000000_0, startTime=1452788989079, finishTime=1452788989401, timeTaken=322, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=245, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=260, MERGE_PHASE_TIME=262, FIRST_EVENT_RECEIVED=258, LAST_EVENT_RECEIVED=258, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788989387, Process_Duration=8, Process_End_Time=1452788989395, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-01-14 16:29:49,404 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788988696_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:49,404 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:49,404 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788988696_0001_1_01_000000_0
2016-01-14 16:29:49,404 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:49,405 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788988696_0001_1][Event:TASK_FINISHED]: vertexName=FE6F7F9372B94E1B832360F2C2A761E7, taskId=task_1452788988696_0001_1_01_000000, startTime=1452788989079, finishTime=1452788989404, timeTaken=325, status=SUCCEEDED, successfulAttemptID=attempt_1452788988696_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=245, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=260, MERGE_PHASE_TIME=262, FIRST_EVENT_RECEIVED=258, LAST_EVENT_RECEIVED=258, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788989387, Process_Duration=8, Process_End_Time=1452788989395, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-01-14 16:29:49,405 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788988696_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:49,405 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788988696_0001_1_01 [FE6F7F9372B94E1B832360F2C2A761E7] : 1
2016-01-14 16:29:49,405 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788988696_0001_1_01 [FE6F7F9372B94E1B832360F2C2A761E7]
2016-01-14 16:29:49,407 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:49,407 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788988696_0001_1][Event:VERTEX_FINISHED]: vertexName=FE6F7F9372B94E1B832360F2C2A761E7, vertexId=vertex_1452788988696_0001_1_01, initRequestedTime=1452788988899, initedTime=1452788988910, startRequestedTime=1452788988936, startedTime=1452788988936, finishTime=1452788989406, timeTaken=470, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=245, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=260, MERGE_PHASE_TIME=262, FIRST_EVENT_RECEIVED=258, LAST_EVENT_RECEIVED=258, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788989387, Process_Duration=8, Process_End_Time=1452788989395, Read_Duration=2, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8, vertexStats=firstTaskStartTime=1452788989079, firstTasksToStart=[ task_1452788988696_0001_1_01_000000 ], lastTaskFinishTime=1452788989401, lastTasksToFinish=[ task_1452788988696_0001_1_01_000000 ], minTaskDuration=322, maxTaskDuration=322, avgTaskDuration=322.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788988696_0001_1_01_000000 ], longestDurationTasks=[ task_1452788988696_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:49,407 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788988696_0001_1_01 [FE6F7F9372B94E1B832360F2C2A761E7] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:49,408 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788988696_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:49,410 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:49,410 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788988696_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788988696_0001_00_000002, stoppedTime=1452788989409, exitStatus=0
2016-01-14 16:29:49,411 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788988696_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:49,411 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788988696_0001_1_01 [FE6F7F9372B94E1B832360F2C2A761E7] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 16:29:49,411 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452788988696_0001_1
2016-01-14 16:29:49,412 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788988696_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452788988696_0001_1
2016-01-14 16:29:49,412 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: E086FDBFFE384C8F9A438BC227036D33 for vertex: vertex_1452788988696_0001_1_01 [FE6F7F9372B94E1B832360F2C2A761E7]
2016-01-14 16:29:49,419 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452788988696_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 16:29:49,419 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788988696_0001_00_000002: 16 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 16:29:49,420 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452788988696_0001_00_000002, Exiting
2016-01-14 16:29:49,420 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788988696_0001_00_000002
2016-01-14 16:29:49,420 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788988696_0001_00_000002
2016-01-14 16:29:49,421 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788988696_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 16:29:49,422 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452788988696_0001_00_000002 completed successfully
2016-01-14 16:29:49,426 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452788988696_0001_1_00 [B07960E9FDFF4523B21ED96B40BAC6C0]
2016-01-14 16:29:49,429 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452788988696_0001_1, queueSize=0
2016-01-14 16:29:49,430 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 16:29:49,430 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788988696_0001_1][Event:DAG_FINISHED]: dagId=dag_1452788988696_0001_1, startTime=1452788988896, finishTime=1452788989426, timeTaken=530, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2442, FILE_BYTES_WRITTEN=376, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=245, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=833093632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=18, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=260, MERGE_PHASE_TIME=262, FIRST_EVENT_RECEIVED=258, LAST_EVENT_RECEIVED=258, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905577978416, Process_Duration=11, Process_End_Time=2905577978427, Read_Duration=4, Tuples_Read=20, Tuples_Written=18, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=8
2016-01-14 16:29:49,430 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452788988696_0001_1 finished with state: SUCCEEDED
2016-01-14 16:29:49,430 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788988696_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 16:29:49,431 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452788988696_0001_00_000002
2016-01-14 16:29:49,432 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788988696_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 16:29:49 Completed Dag: dag_1452788988696_0001_1
2016-01-14 16:29:49,435 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452788988696_0001_1, dagState=SUCCEEDED
2016-01-14 16:29:49,436 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788988696_0001_1_post
2016-01-14 16:29:49,436 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:49,436 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 16:29:49,440 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...leRelative/simplerelative, applicationId=application_1452788988696_0001
2016-01-14 16:29:49,443 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 16:29:49,444 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 16:29:49,444 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 16:29:49,444 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 16:29:49,444 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 16:29:49,446 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 16:29:49,447 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 16:29:49,448 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 16:29:49,448 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 16:29:49,448 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 16:29:49,449 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 16:29:49,450 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 16:29:49,450 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 16:29:49,456 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/F475903C5431465B8832A106F38A02B6/CC3D0CDACE4D4DE2B84BCD1B6C623BBF/.tez/application_1452788988696_0001
2016-01-14 16:29:49,457 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:00.781
2016-01-14 16:29:49,513 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative
2016-01-14 16:29:49,647 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 16:29:49,647 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 16:29:49,704 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.027
2016-01-14 16:29:49,727 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.050
2016-01-14 16:29:49,727 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 16:29:49,727 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 16:29:49,728 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 16:29:49,744 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt
2016-01-14 16:29:49,760 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult
2016-01-14 16:29:49,812 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 16:29:49,812 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 16:29:49,813 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult"]
2016-01-14 16:29:49,813 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 16:29:49,813 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 16:29:49,813 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 16:29:49,814 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...SimpleResult/simpleresult
2016-01-14 16:29:49,814 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 16:29:49,827 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 16:29:49,831 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:49,832 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 16:29:49,832 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 16:29:49,841 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8BD7F1823841430FBE169D71A28E269B/D6F454BB1FDE4E70936C0FFC41DE6858/.tez/application_1452788989832_0001 doesn't exist and is created
2016-01-14 16:29:49,858 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 16:29:49,858 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 16:29:49,859 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8BD7F1823841430FBE169D71A28E269B/D6F454BB1FDE4E70936C0FFC41DE6858/.tez/application_1452788989832_0001
2016-01-14 16:29:49,863 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452788989832_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:49,880 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 16:29:49,882 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 16:29:49,884 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 16:29:49,885 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 16:29:49,885 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8BD7F1823841430FBE169D71A28E269B/D6F454BB1FDE4E70936C0FFC41DE6858/.tez/application_1452788989832_0001/localmode-log-dir
2016-01-14 16:29:49,886 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8BD7F1823841430FBE169D71A28E269B/D6F454BB1FDE4E70936C0FFC41DE6858/.tez/application_1452788989832_0001/localmode-log-dir/history.txt.appattempt_1452788989832_0001_000000, maxErrors=10
2016-01-14 16:29:49,886 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 16:29:49,886 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452788989832_0001_000000, appSubmitTime=1452788989863, launchTime=1452788989863
2016-01-14 16:29:49,892 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 16:29:49,892 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 16:29:49,896 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:59213
2016-01-14 16:29:49,897 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 16:29:49,897 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 16:29:49,902 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 16:29:49,905 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452788989832_0001_000000, startTime=1452788989905
2016-01-14 16:29:49,905 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 16:29:49,905 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 16:29:49,959 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 16:29:49,959 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 16:29:49,959 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...SimpleResult/simpleresult, applicationId=application_1452788989832_0001, dagName=[8BD7F1823841430FBE169D71A28E269B/D6F454BB1FDE4E70936C0FFC41DE6858] test/(1/1) ...SimpleResult/simpleresult
2016-01-14 16:29:49,978 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [8BD7F1823841430FBE169D71A28E269B/D6F454BB1FDE4E70936C0FFC41DE6858] test/(1/1) ...SimpleResult/simpleresult
2016-01-14 16:29:49,980 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452788989832_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8BD7F1823841430FBE169D71A28E269B/D6F454BB1FDE4E70936C0FFC41DE6858/.tez/application_1452788989832_0001/localmode-log-dir/dag_1452788989832_0001_1.dot
2016-01-14 16:29:49,981 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8BD7F1823841430FBE169D71A28E269B/D6F454BB1FDE4E70936C0FFC41DE6858/.tez/application_1452788989832_0001/localmode-log-dir/dag_1452788989832_0001_1-tez-dag.pb.txt
2016-01-14 16:29:49,991 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788989832_0001_1
2016-01-14 16:29:49,991 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:49,991 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [8BD7F1823841430FBE169D71A28E269B/D6F454BB1FDE4E70936C0FFC41DE6858] test/(1/1) ...SimpleResult/simpleresult
2016-01-14 16:29:49 Running Dag: dag_1452788989832_0001_1
2016-01-14 16:29:50,004 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 16:29:50,005 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788989832_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452788989832_0001_1, submitTime=1452788989978
2016-01-14 16:29:50,006 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FFC4055B53B9422E833F1AA38B82F80D
2016-01-14 16:29:50,006 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 16:29:50,006 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 16:29:50,007 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788989832_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452788989832_0001_1, initTime=1452788990005
2016-01-14 16:29:50,007 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788989832_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 16:29:50,007 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788989832_0001_1][Event:DAG_STARTED]: dagID=dag_1452788989832_0001_1, startTime=1452788990007
2016-01-14 16:29:50,008 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 16:29:50,008 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788989832_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 16:29:50,008 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 16:29:50,008 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A9F6DCF797364DE58718CDEF6C6DE7C2 : {435CDC77779C489B9718352A39D31DE3={InputName=435CDC77779C489B9718352A39D31DE3}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 16:29:50,008 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 435CDC77779C489B9718352A39D31DE3, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 16:29:50,008 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452788989832_0001_1_00 [A9F6DCF797364DE58718CDEF6C6DE7C2]
2016-01-14 16:29:50,008 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452788989832_0001_1_00 [A9F6DCF797364DE58718CDEF6C6DE7C2]
2016-01-14 16:29:50,008 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452788989832_0001_1_00 [A9F6DCF797364DE58718CDEF6C6DE7C2]
2016-01-14 16:29:50,009 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452788989832_0001_1_00 [A9F6DCF797364DE58718CDEF6C6DE7C2]. Starting root input initializers: 1
2016-01-14 16:29:50,009 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...SimpleResult/simpleresult, applicationId=application_1452788989832_0001, dagName=[8BD7F1823841430FBE169D71A28E269B/D6F454BB1FDE4E70936C0FFC41DE6858] test/(1/1) ...SimpleResult/simpleresult
2016-01-14 16:29:50,010 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788989832_0001_1_00 [A9F6DCF797364DE58718CDEF6C6DE7C2] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 16:29:50,010 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452788989832_0001_1_01 [FFC4055B53B9422E833F1AA38B82F80D]
2016-01-14 16:29:50,012 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 435CDC77779C489B9718352A39D31DE3 on vertex vertex_1452788989832_0001_1_00 [A9F6DCF797364DE58718CDEF6C6DE7C2]
2016-01-14 16:29:50,013 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 16:29:50,013 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452788989832_0001_1_01 [FFC4055B53B9422E833F1AA38B82F80D]
2016-01-14 16:29:50,013 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452788989832_0001_1_01 [FFC4055B53B9422E833F1AA38B82F80D]
2016-01-14 16:29:50,013 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 16:29:50,013 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452788989832_0001_1_01 [FFC4055B53B9422E833F1AA38B82F80D]
2016-01-14 16:29:50,013 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=35B69E76299D4097A6C62F97BE592D12, vertexId=vertex_1452788989832_0001_1_01 [FFC4055B53B9422E833F1AA38B82F80D], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 16:29:50,014 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=35B69E76299D4097A6C62F97BE592D12, vertexId=vertex_1452788989832_0001_1_01 [FFC4055B53B9422E833F1AA38B82F80D]
2016-01-14 16:29:50,015 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 435CDC77779C489B9718352A39D31DE3 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 16:29:50,017 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 16:29:50,017 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=35B69E76299D4097A6C62F97BE592D12, vertexName=FFC4055B53B9422E833F1AA38B82F80D, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 16:29:50,018 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=35B69E76299D4097A6C62F97BE592D12, vertexId=vertex_1452788989832_0001_1_01 [FFC4055B53B9422E833F1AA38B82F80D]
2016-01-14 16:29:50,018 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 16:29:50,020 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:50,022 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788989832_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FFC4055B53B9422E833F1AA38B82F80D, vertexId=vertex_1452788989832_0001_1_01, initRequestedTime=1452788990010, initedTime=1452788990020, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 16:29:50,022 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788989832_0001_1_01 [FFC4055B53B9422E833F1AA38B82F80D] transitioned from NEW to INITED due to event V_INIT
2016-01-14 16:29:50,023 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-01-14 16:29:50,023 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 435CDC77779C489B9718352A39D31DE3 on vertex vertex_1452788989832_0001_1_00 [A9F6DCF797364DE58718CDEF6C6DE7C2]
2016-01-14 16:29:50,023 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788989832_0001_1_00 [A9F6DCF797364DE58718CDEF6C6DE7C2] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:50,024 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {435CDC77779C489B9718352A39D31DE3=forAllWorkUnits=true, update=[1]}
2016-01-14 16:29:50,024 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452788989832_0001_1_00 [A9F6DCF797364DE58718CDEF6C6DE7C2] parallelism set to 1
2016-01-14 16:29:50,024 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788989832_0001_1_00 [A9F6DCF797364DE58718CDEF6C6DE7C2] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:50,025 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788989832_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788989832_0001_1_00, eventCount=1
2016-01-14 16:29:50,026 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:50,026 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788989832_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A9F6DCF797364DE58718CDEF6C6DE7C2, vertexId=vertex_1452788989832_0001_1_00, initRequestedTime=1452788990008, initedTime=1452788990025, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 16:29:50,026 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788989832_0001_1_00 [A9F6DCF797364DE58718CDEF6C6DE7C2] with distanceFromRoot: 0
2016-01-14 16:29:50,027 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788989832_0001_1_00 [A9F6DCF797364DE58718CDEF6C6DE7C2] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 16:29:50,027 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A9F6DCF797364DE58718CDEF6C6DE7C2
2016-01-14 16:29:50,027 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788989832_0001_1_00 [A9F6DCF797364DE58718CDEF6C6DE7C2]
2016-01-14 16:29:50,028 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:50,029 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788989832_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788989832_0001_1_00, startRequestedTime=1452788990027, startedTime=1452788990027
2016-01-14 16:29:50,029 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788989832_0001_1_00 [A9F6DCF797364DE58718CDEF6C6DE7C2] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:50,030 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:50,030 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788989832_0001_1][Event:TASK_STARTED]: vertexName=A9F6DCF797364DE58718CDEF6C6DE7C2, taskId=task_1452788989832_0001_1_00_000000, scheduledTime=1452788990029, launchTime=1452788990029
2016-01-14 16:29:50,030 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788989832_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:50,030 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452788989832_0001, with dag id: dag_1452788989832_0001_1
2016-01-14 16:29:50,031 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452788989832_0001_1_00 for vertex: vertex_1452788989832_0001_1_01 [FFC4055B53B9422E833F1AA38B82F80D] numStartedSources: 1 numSources: 1
2016-01-14 16:29:50,031 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788989832_0001_1_01 [FFC4055B53B9422E833F1AA38B82F80D] with distanceFromRoot: 1
2016-01-14 16:29:50,032 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788989832_0001_1_00 [A9F6DCF797364DE58718CDEF6C6DE7C2]
2016-01-14 16:29:50,032 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: A9F6DCF797364DE58718CDEF6C6DE7C2 in vertex: FFC4055B53B9422E833F1AA38B82F80D
2016-01-14 16:29:50,033 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: FFC4055B53B9422E833F1AA38B82F80D with 1 source tasks and 1 pending tasks
2016-01-14 16:29:50,034 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:50,034 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788989832_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788989832_0001_1_01, startRequestedTime=1452788990032, startedTime=1452788990032
2016-01-14 16:29:50,034 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788989832_0001_1_01 [FFC4055B53B9422E833F1AA38B82F80D] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:50,035 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8BD7F1823841430FBE169D71A28E269B/D6F454BB1FDE4E70936C0FFC41DE6858] test/(1/1) ...SimpleResult/simpleresult, VertexName: A9F6DCF797364DE58718CDEF6C6DE7C2, VertexParallelism: 1, TaskAttemptID:attempt_1452788989832_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=435CDC77779C489B9718352A39D31DE3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FFC4055B53B9422E833F1AA38B82F80D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 16:29:50,035 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788989832_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:50,035 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:50,037 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 16:29:50,038 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 16:29:50,039 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 16:29:50,039 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788989832_0001_00_000001
2016-01-14 16:29:50,039 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788989832_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:50,040 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788989832_0001_00_000001
2016-01-14 16:29:50,041 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 16:29:50,041 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788989832_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:50,042 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:50,042 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788989832_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788989832_0001_00_000001, launchTime=1452788990041
2016-01-14 16:29:50,043 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788989832_0001_00_000001
2016-01-14 16:29:50,043 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788989832_0001_1_00_000000_0] to container: [container_1452788989832_0001_00_000001]
2016-01-14 16:29:50,043 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788989832_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:50,044 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788989832_0001_00_000001 given task: attempt_1452788989832_0001_1_00_000000_0
2016-01-14 16:29:50,044 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788989832_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788989832_0001_1_00_000000_0
2016-01-14 16:29:50,044 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788989832_0001_1_00_000000_0] started. Is using containerId: [container_1452788989832_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 16:29:50,044 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788989832_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A9F6DCF797364DE58718CDEF6C6DE7C2, taskAttemptId=attempt_1452788989832_0001_1_00_000000_0, startTime=1452788990044, containerId=container_1452788989832_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788989832_0001_00_000001/teamcity, completedLogs=
2016-01-14 16:29:50,044 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788989832_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:50,045 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788989832_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:50,045 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:50 Starting to run new task attempt: attempt_1452788989832_0001_1_00_000000_0
2016-01-14 16:29:50,046 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788989832_0001_1_00_000000_0
2016-01-14 16:29:50,046 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:50,046 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:50,046 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:50,046 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:50,047 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:50,047 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:50,047 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8BD7F1823841430FBE169D71A28E269B/D6F454BB1FDE4E70936C0FFC41DE6858] test/(1/1) ...SimpleResult/simpleresult, VertexName: A9F6DCF797364DE58718CDEF6C6DE7C2, VertexParallelism: 1, TaskAttemptID:attempt_1452788989832_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=435CDC77779C489B9718352A39D31DE3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FFC4055B53B9422E833F1AA38B82F80D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 16:29:50,047 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:50,048 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788989832_0001_1_00_000000_0
2016-01-14 16:29:50,048 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:50,048 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:50,049 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:50,050 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FFC4055B53B9422E833F1AA38B82F80D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 16:29:50,051 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:50,053 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FFC4055B53B9422E833F1AA38B82F80D
2016-01-14 16:29:50,053 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=435CDC77779C489B9718352A39D31DE3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 16:29:50,054 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:50,054 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:50,055 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 16:29:50,055 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: FFC4055B53B9422E833F1AA38B82F80D
2016-01-14 16:29:50,055 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FFC4055B53B9422E833F1AA38B82F80D
2016-01-14 16:29:50,054 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 435CDC77779C489B9718352A39D31DE3
2016-01-14 16:29:50,056 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 435CDC77779C489B9718352A39D31DE3
2016-01-14 16:29:50,056 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788989832_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 16:29:50,059 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 16:29:50,059 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 435CDC77779C489B9718352A39D31DE3
2016-01-14 16:29:50,059 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 435CDC77779C489B9718352A39D31DE3
2016-01-14 16:29:50,065 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A9F6DCF797364DE58718CDEF6C6DE7C2, ordinal: 0
2016-01-14 16:29:50,065 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A9F6DCF797364DE58718CDEF6C6DE7C2, mem on start (mb), free: 244, total: 357, max: 672
2016-01-14 16:29:50,065 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:50,066 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:50,066 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:50,066 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:50,066 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:50,067 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 16:29:50,067 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 16:29:50,067 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:50,068 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 16:29:50,068 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 16:29:50,068 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FFC4055B53B9422E833F1AA38B82F80D, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 16:29:50,068 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 435CDC77779C489B9718352A39D31DE3, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 16:29:50,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:50,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 435CDC77779C489B9718352A39D31DE3 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:50,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 435CDC77779C489B9718352A39D31DE3
2016-01-14 16:29:50,069 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:50,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 435CDC77779C489B9718352A39D31DE3
2016-01-14 16:29:50,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 16:29:50,070 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:50,070 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:50,070 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788989832_0001_1_00_000000_0
2016-01-14 16:29:50,070 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 16:29:50,071 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@fd99cca
2016-01-14 16:29:50,072 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@fd99cca
2016-01-14 16:29:50,072 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 16:29:50,073 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 16:29:50,073 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 16:29:50,076 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"] streamed: true, id: 435CDC77779C489B9718352A39D31DE3
2016-01-14 16:29:50,076 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: E938E277590944628440349BF1C16CBE
2016-01-14 16:29:50,076 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] E938E277590944628440349BF1C16CBE
2016-01-14 16:29:50,077 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@7e13c53; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@543f712f; comparator=cascading.tuple.hadoop.util.TupleComparator@111fefdf; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 16:29:50,077 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 16:29:50,077 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 16:29:50,077 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 16:29:50,091 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 16:29:50,091 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 16:29:50,091 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 16:29:50,091 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 16:29:50,091 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 16:29:50,174 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 16:29:50,174 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:50,175 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A9F6DCF797364DE58718CDEF6C6DE7C2, all 1 inputs ready in: 00:00:00.000
2016-01-14 16:29:50,178 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788989832_0001_1_00_000000_0
2016-01-14 16:29:50,178 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A9F6DCF797364DE58718CDEF6C6DE7C2, mem on close (mb), free: 139, total: 357, max: 672
2016-01-14 16:29:50,178 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 16:29:50,178 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 16:29:50,178 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 22; bufvoid = 104857600
2016-01-14 16:29:50,179 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214392(104857568); length = 5/6553600
2016-01-14 16:29:50,181 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 16:29:50 Completed running task attempt: attempt_1452788989832_0001_1_00_000000_0
2016-01-14 16:29:50,187 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788989832_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:50,187 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788989832_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788989832_0001_1_00, eventCount=1
2016-01-14 16:29:50,187 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=68, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452788990175, Process_Duration=3, Process_End_Time=1452788990178, Read_Duration=2, Tuples_Read=5, Tuples_Written=2, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-01-14 16:29:50,188 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:50,188 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:50,188 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788989832_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A9F6DCF797364DE58718CDEF6C6DE7C2, taskAttemptId=attempt_1452788989832_0001_1_00_000000_0, startTime=1452788990044, finishTime=1452788990188, timeTaken=144, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=68, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788990175, Process_Duration=3, Process_End_Time=1452788990178, Read_Duration=2, Tuples_Read=5, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 16:29:50,188 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:50,189 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788989832_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:50,189 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788989832_0001_00_000001
2016-01-14 16:29:50,189 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788989832_0001_1_00_000000_0
2016-01-14 16:29:50,189 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:50,190 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788989832_0001_1][Event:TASK_FINISHED]: vertexName=A9F6DCF797364DE58718CDEF6C6DE7C2, taskId=task_1452788989832_0001_1_00_000000, startTime=1452788990044, finishTime=1452788990189, timeTaken=145, status=SUCCEEDED, successfulAttemptID=attempt_1452788989832_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=68, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788990175, Process_Duration=3, Process_End_Time=1452788990178, Read_Duration=2, Tuples_Read=5, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 16:29:50,191 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788989832_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:50,191 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788989832_0001_1_00 [A9F6DCF797364DE58718CDEF6C6DE7C2] : 1
2016-01-14 16:29:50,191 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788989832_0001_1_00 [A9F6DCF797364DE58718CDEF6C6DE7C2]
2016-01-14 16:29:50,192 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:50,194 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788989832_0001_1][Event:VERTEX_FINISHED]: vertexName=A9F6DCF797364DE58718CDEF6C6DE7C2, vertexId=vertex_1452788989832_0001_1_00, initRequestedTime=1452788990008, initedTime=1452788990025, startRequestedTime=1452788990027, startedTime=1452788990027, finishTime=1452788990191, timeTaken=164, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=68, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788990175, Process_Duration=3, Process_End_Time=1452788990178, Read_Duration=2, Tuples_Read=5, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1452788990044, firstTasksToStart=[ task_1452788989832_0001_1_00_000000 ], lastTaskFinishTime=1452788990188, lastTasksToFinish=[ task_1452788989832_0001_1_00_000000 ], minTaskDuration=144, maxTaskDuration=144, avgTaskDuration=144.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788989832_0001_1_00_000000 ], longestDurationTasks=[ task_1452788989832_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:50,194 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788989832_0001_1_00 [A9F6DCF797364DE58718CDEF6C6DE7C2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:50,194 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788989832_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:50,194 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788989832_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788989832_0001_00_000001, stoppedTime=1452788990194, exitStatus=0
2016-01-14 16:29:50,195 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788989832_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:50,194 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:50,195 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452788989832_0001_1_01 [FFC4055B53B9422E833F1AA38B82F80D] attempt: attempt_1452788989832_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 16:29:50,196 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: FFC4055B53B9422E833F1AA38B82F80D
2016-01-14 16:29:50,196 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788989832_0001_1_01 [FFC4055B53B9422E833F1AA38B82F80D]
2016-01-14 16:29:50,196 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:50,197 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788989832_0001_1_00 [A9F6DCF797364DE58718CDEF6C6DE7C2] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 16:29:50,197 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452788989832_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 16:29:50,198 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788989832_0001_1][Event:TASK_STARTED]: vertexName=FFC4055B53B9422E833F1AA38B82F80D, taskId=task_1452788989832_0001_1_01_000000, scheduledTime=1452788990197, launchTime=1452788990197
2016-01-14 16:29:50,198 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:50,198 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788989832_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:50,198 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788989832_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 16:29:50,198 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788989832_0001_1_01 [FFC4055B53B9422E833F1AA38B82F80D]
2016-01-14 16:29:50,199 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8BD7F1823841430FBE169D71A28E269B/D6F454BB1FDE4E70936C0FFC41DE6858] test/(1/1) ...SimpleResult/simpleresult, VertexName: FFC4055B53B9422E833F1AA38B82F80D, VertexParallelism: 1, TaskAttemptID:attempt_1452788989832_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A9F6DCF797364DE58718CDEF6C6DE7C2, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=35B69E76299D4097A6C62F97BE592D12, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:50,200 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788989832_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:50,200 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:50,200 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452788989832_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 16:29:50,200 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788989832_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 16:29:50,201 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788989832_0001_00_000002
2016-01-14 16:29:50,201 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788989832_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:50,201 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452788989832_0001_00_000001, Exiting
2016-01-14 16:29:50,201 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788989832_0001_00_000001
2016-01-14 16:29:50,201 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788989832_0001_00_000001
2016-01-14 16:29:50,205 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788989832_0001_00_000002
2016-01-14 16:29:50,206 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788989832_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 16:29:50,206 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452788989832_0001_00_000001 completed successfully
2016-01-14 16:29:50,206 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788989832_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 16:29:50,206 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 16:29:50,208 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788989832_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:50,208 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:50,208 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788989832_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788989832_0001_00_000002, launchTime=1452788990208
2016-01-14 16:29:50,209 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788989832_0001_00_000002
2016-01-14 16:29:50,210 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788989832_0001_1_01_000000_0] to container: [container_1452788989832_0001_00_000002]
2016-01-14 16:29:50,210 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788989832_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:50,210 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788989832_0001_00_000002 given task: attempt_1452788989832_0001_1_01_000000_0
2016-01-14 16:29:50,211 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788989832_0001_1_01_000000_0] started. Is using containerId: [container_1452788989832_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 16:29:50,211 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788989832_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788989832_0001_1_01_000000_0
2016-01-14 16:29:50,211 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:50,211 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788989832_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FFC4055B53B9422E833F1AA38B82F80D, taskAttemptId=attempt_1452788989832_0001_1_01_000000_0, startTime=1452788990211, containerId=container_1452788989832_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788989832_0001_00_000002/teamcity, completedLogs=
2016-01-14 16:29:50,211 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788989832_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:50,212 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788989832_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:50 Starting to run new task attempt: attempt_1452788989832_0001_1_01_000000_0
2016-01-14 16:29:50,213 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788989832_0001_1_01_000000_0
2016-01-14 16:29:50,214 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:50,214 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:50,214 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:50,214 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:50,215 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:50,215 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:50,215 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8BD7F1823841430FBE169D71A28E269B/D6F454BB1FDE4E70936C0FFC41DE6858] test/(1/1) ...SimpleResult/simpleresult, VertexName: FFC4055B53B9422E833F1AA38B82F80D, VertexParallelism: 1, TaskAttemptID:attempt_1452788989832_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A9F6DCF797364DE58718CDEF6C6DE7C2, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=35B69E76299D4097A6C62F97BE592D12, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:50,215 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:50,216 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788989832_0001_1_01_000000_0
2016-01-14 16:29:50,216 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:50,217 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:50,217 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788989832_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 16:29:50,217 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A9F6DCF797364DE58718CDEF6C6DE7C2, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 16:29:50,218 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=35B69E76299D4097A6C62F97BE592D12, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 16:29:50,217 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:50,218 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:50,218 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:50,218 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A9F6DCF797364DE58718CDEF6C6DE7C2
2016-01-14 16:29:50,219 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 35B69E76299D4097A6C62F97BE592D12
2016-01-14 16:29:50,219 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 16:29:50,220 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 35B69E76299D4097A6C62F97BE592D12
2016-01-14 16:29:50,220 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 16:29:50,220 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 16:29:50,221 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: A9F6DCF797364DE58718CDEF6C6DE7C2
2016-01-14 16:29:50,221 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A9F6DCF797364DE58718CDEF6C6DE7C2
2016-01-14 16:29:50,223 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:50,233 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 16:29:50,233 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 35B69E76299D4097A6C62F97BE592D12
2016-01-14 16:29:50,234 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FFC4055B53B9422E833F1AA38B82F80D, ordinal: 1
2016-01-14 16:29:50,234 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FFC4055B53B9422E833F1AA38B82F80D, mem on start (mb), free: 133, total: 357, max: 672
2016-01-14 16:29:50,234 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:50,235 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:50,235 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:50,235 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:50,235 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:50,236 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 16:29:50,236 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:50,236 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 16:29:50,237 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 16:29:50,237 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 16:29:50,238 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 35B69E76299D4097A6C62F97BE592D12, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 16:29:50,238 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A9F6DCF797364DE58718CDEF6C6DE7C2, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 16:29:50,238 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:50,238 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A9F6DCF797364DE58718CDEF6C6DE7C2 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:50,238 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A9F6DCF797364DE58718CDEF6C6DE7C2
2016-01-14 16:29:50,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 16:29:50,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:50,240 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 16:29:50,240 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 16:29:50,241 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: A9F6DCF797364DE58718CDEF6C6DE7C2 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 16:29:50,241 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 16:29:50,241 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 16:29:50,241 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 16:29:50,244 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A9F6DCF797364DE58718CDEF6C6DE7C2
2016-01-14 16:29:50,244 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:50,245 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788989832_0001_1_01_000000_0
2016-01-14 16:29:50,245 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452788989832_0001_1_00_000000_0_10039, runDuration: 132261, ]
2016-01-14 16:29:50,247 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452788989832_0001_1_00_000000_0_10039] to DISK_DIRECT, CompressedSize=36, DecompressedSize=32,EndTime=1452788990247, TimeTaken=1, Rate=0.03 MB/s
2016-01-14 16:29:50,247 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : A9F6DCF797364DE58718CDEF6C6DE7C2
2016-01-14 16:29:50,247 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 16:29:50,247 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-01-14 16:29:50,247 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [A9F6DCF797364DE58718CDEF6C6DE7C2] #7 in 1ms
2016-01-14 16:29:50,248 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [A9F6DCF797364DE58718CDEF6C6DE7C2] #7, status:false, isInterrupted:false
2016-01-14 16:29:50,248 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: E938E277590944628440349BF1C16CBE
2016-01-14 16:29:50,249 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult"], id: 35B69E76299D4097A6C62F97BE592D12
2016-01-14 16:29:50,248 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 16:29:50,249 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 36 bytes from disk
2016-01-14 16:29:50,249 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult"]
2016-01-14 16:29:50,250 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] E938E277590944628440349BF1C16CBE, for 1 inputs
2016-01-14 16:29:50,250 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 16:29:50,250 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 16:29:50,251 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 17 bytes
2016-01-14 16:29:50,251 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : A9F6DCF797364DE58718CDEF6C6DE7C2
2016-01-14 16:29:50,251 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FFC4055B53B9422E833F1AA38B82F80D, all 1 inputs ready in: 00:00:00.001
2016-01-14 16:29:50,251 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 16:29:50,252 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@aaadc72; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@576ec5d6
2016-01-14 16:29:50,253 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 16:29:50,254 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 16:29:50,254 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452788989832_0001_1_01_000000_0
2016-01-14 16:29:50,255 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452788989832_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 16:29:50,256 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788989832_0001_1_01_000000_0
2016-01-14 16:29:50,256 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FFC4055B53B9422E833F1AA38B82F80D, mem on close (mb), free: 130, total: 357, max: 672
2016-01-14 16:29:50,256 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: A9F6DCF797364DE58718CDEF6C6DE7C2
2016-01-14 16:29:50,256 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 16:29:50,256 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 16:29:50,257 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 16:29:50,257 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 16:29:50,257 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 16:29:50 Completed running task attempt: attempt_1452788989832_0001_1_01_000000_0
2016-01-14 16:29:50,258 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788989832_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:50,259 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=68, FILE_BYTES_WRITTEN=24, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452788990251, Process_Duration=4, Process_End_Time=1452788990255, Read_Duration=0, Tuples_Read=2, Tuples_Written=2, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=2]]
2016-01-14 16:29:50,259 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788989832_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FFC4055B53B9422E833F1AA38B82F80D, taskAttemptId=attempt_1452788989832_0001_1_01_000000_0, startTime=1452788990211, finishTime=1452788990259, timeTaken=48, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=68, FILE_BYTES_WRITTEN=24, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788990251, Process_Duration=4, Process_End_Time=1452788990255, Read_Duration=0, Tuples_Read=2, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=2
2016-01-14 16:29:50,259 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788989832_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:50,260 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:50,259 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:50,260 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:50,260 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:50,260 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788989832_0001_1_01_000000_0
2016-01-14 16:29:50,261 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788989832_0001_1][Event:TASK_FINISHED]: vertexName=FFC4055B53B9422E833F1AA38B82F80D, taskId=task_1452788989832_0001_1_01_000000, startTime=1452788990211, finishTime=1452788990260, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1452788989832_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=68, FILE_BYTES_WRITTEN=24, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788990251, Process_Duration=4, Process_End_Time=1452788990255, Read_Duration=0, Tuples_Read=2, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=2
2016-01-14 16:29:50,261 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788989832_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:50,261 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:50,262 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788989832_0001_1_01 [FFC4055B53B9422E833F1AA38B82F80D] : 1
2016-01-14 16:29:50,262 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788989832_0001_1_01 [FFC4055B53B9422E833F1AA38B82F80D]
2016-01-14 16:29:50,262 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788989832_0001_00_000002
2016-01-14 16:29:50,264 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788989832_0001_1][Event:VERTEX_FINISHED]: vertexName=FFC4055B53B9422E833F1AA38B82F80D, vertexId=vertex_1452788989832_0001_1_01, initRequestedTime=1452788990010, initedTime=1452788990020, startRequestedTime=1452788990032, startedTime=1452788990032, finishTime=1452788990263, timeTaken=231, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=68, FILE_BYTES_WRITTEN=24, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788990251, Process_Duration=4, Process_End_Time=1452788990255, Read_Duration=0, Tuples_Read=2, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=2, vertexStats=firstTaskStartTime=1452788990211, firstTasksToStart=[ task_1452788989832_0001_1_01_000000 ], lastTaskFinishTime=1452788990259, lastTasksToFinish=[ task_1452788989832_0001_1_01_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788989832_0001_1_01_000000 ], longestDurationTasks=[ task_1452788989832_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:50,264 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788989832_0001_1_01 [FFC4055B53B9422E833F1AA38B82F80D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:50,264 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788989832_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:50,264 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788989832_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788989832_0001_00_000002, stoppedTime=1452788990264, exitStatus=0
2016-01-14 16:29:50,264 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788989832_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:50,264 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788989832_0001_1_01 [FFC4055B53B9422E833F1AA38B82F80D] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 16:29:50,265 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452788989832_0001_1
2016-01-14 16:29:50,265 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788989832_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452788989832_0001_1
2016-01-14 16:29:50,265 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452788989832_0001_1_00 [A9F6DCF797364DE58718CDEF6C6DE7C2]
2016-01-14 16:29:50,265 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 35B69E76299D4097A6C62F97BE592D12 for vertex: vertex_1452788989832_0001_1_01 [FFC4055B53B9422E833F1AA38B82F80D]
2016-01-14 16:29:50,265 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:50,269 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:50,273 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452788989832_0001_1, queueSize=0
2016-01-14 16:29:50,274 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452788989832_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 16:29:50,274 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788989832_0001_1][Event:DAG_FINISHED]: dagId=dag_1452788989832_0001_1, startTime=1452788990007, finishTime=1452788990272, timeTaken=265, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=130, FILE_BYTES_WRITTEN=92, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=4, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=749731840, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=4, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905577980426, Process_Duration=7, Process_End_Time=2905577980433, Read_Duration=2, Tuples_Read=7, Tuples_Written=4, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=2
2016-01-14 16:29:50,274 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452788989832_0001_1 finished with state: SUCCEEDED
2016-01-14 16:29:50,274 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788989832_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 16:29:50,274 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452788989832_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 16:29:50,274 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788989832_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 16:29:50,274 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 16:29:50,275 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452788989832_0001_00_000002, Exiting
2016-01-14 16:29:50,276 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788989832_0001_00_000002
2016-01-14 16:29:50,276 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788989832_0001_00_000002
2016-01-14 16:29:50,276 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788989832_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 16:29:50,276 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452788989832_0001_00_000002 completed successfully
2016-01-14 16:29:50 Completed Dag: dag_1452788989832_0001_1
2016-01-14 16:29:50,277 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452788989832_0001_1, dagState=SUCCEEDED
2016-01-14 16:29:50,277 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788989832_0001_1_post
2016-01-14 16:29:50,277 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:50,277 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 16:29:50,278 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788989832_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 16:29:50,278 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788989832_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 16:29:50,284 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...SimpleResult/simpleresult, applicationId=application_1452788989832_0001
2016-01-14 16:29:50,287 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 16:29:50,288 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 16:29:50,288 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 16:29:50,288 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 16:29:50,288 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 16:29:50,295 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 16:29:50,295 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 16:29:50,295 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 16:29:50,296 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 16:29:50,296 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 16:29:50,297 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 16:29:50,299 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 16:29:50,299 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 16:29:50,306 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8BD7F1823841430FBE169D71A28E269B/D6F454BB1FDE4E70936C0FFC41DE6858/.tez/application_1452788989832_0001
2016-01-14 16:29:50,306 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:00.494
2016-01-14 16:29:50,343 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult
2016-01-14 16:29:50,366 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 16:29:50,366 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 16:29:50,403 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-01-14 16:29:50,409 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-01-14 16:29:50,410 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 16:29:50,410 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 16:29:50,410 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 16:29:50,426 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-01-14 16:29:50,426 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall
2016-01-14 16:29:50,435 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 16:29:50,435 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 16:29:50,436 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall"]
2016-01-14 16:29:50,436 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 16:29:50,436 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 16:29:50,436 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 16:29:50,437 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...m/testFilterAll/filterall
2016-01-14 16:29:50,437 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 16:29:50,445 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 16:29:50,445 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 16:29:50,450 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 16:29:50,454 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:50,455 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 16:29:50,455 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 16:29:50,466 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/75BDD423698248BA91D8AA472D13AC05/E385F60660144B8F9DBAFAA6D72A8978/.tez/application_1452788990455_0001 doesn't exist and is created
2016-01-14 16:29:50,484 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 16:29:50,484 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 16:29:50,484 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/75BDD423698248BA91D8AA472D13AC05/E385F60660144B8F9DBAFAA6D72A8978/.tez/application_1452788990455_0001
2016-01-14 16:29:50,488 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452788990455_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:50,498 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 16:29:50,499 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 16:29:50,500 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 16:29:50,500 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 16:29:50,501 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/75BDD423698248BA91D8AA472D13AC05/E385F60660144B8F9DBAFAA6D72A8978/.tez/application_1452788990455_0001/localmode-log-dir
2016-01-14 16:29:50,501 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/75BDD423698248BA91D8AA472D13AC05/E385F60660144B8F9DBAFAA6D72A8978/.tez/application_1452788990455_0001/localmode-log-dir/history.txt.appattempt_1452788990455_0001_000000, maxErrors=10
2016-01-14 16:29:50,501 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 16:29:50,501 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452788990455_0001_000000, appSubmitTime=1452788990487, launchTime=1452788990487
2016-01-14 16:29:50,504 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 16:29:50,507 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 16:29:50,508 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 16:29:50,509 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 16:29:50,511 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:32922
2016-01-14 16:29:50,513 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 16:29:50,515 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452788990455_0001_000000, startTime=1452788990515
2016-01-14 16:29:50,515 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 16:29:50,515 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 16:29:50,584 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 16:29:50,584 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 16:29:50,584 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...m/testFilterAll/filterall, applicationId=application_1452788990455_0001, dagName=[75BDD423698248BA91D8AA472D13AC05/E385F60660144B8F9DBAFAA6D72A8978] test/(1/1) ...m/testFilterAll/filterall
2016-01-14 16:29:50,603 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [75BDD423698248BA91D8AA472D13AC05/E385F60660144B8F9DBAFAA6D72A8978] test/(1/1) ...m/testFilterAll/filterall
2016-01-14 16:29:50,604 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452788990455_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/75BDD423698248BA91D8AA472D13AC05/E385F60660144B8F9DBAFAA6D72A8978/.tez/application_1452788990455_0001/localmode-log-dir/dag_1452788990455_0001_1.dot
2016-01-14 16:29:50,605 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/75BDD423698248BA91D8AA472D13AC05/E385F60660144B8F9DBAFAA6D72A8978/.tez/application_1452788990455_0001/localmode-log-dir/dag_1452788990455_0001_1-tez-dag.pb.txt
2016-01-14 16:29:50,613 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788990455_0001_1
2016-01-14 16:29:50,613 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:50,613 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [75BDD423698248BA91D8AA472D13AC05/E385F60660144B8F9DBAFAA6D72A8978] test/(1/1) ...m/testFilterAll/filterall
2016-01-14 16:29:50 Running Dag: dag_1452788990455_0001_1
2016-01-14 16:29:50,626 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788990455_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452788990455_0001_1, submitTime=1452788990603
2016-01-14 16:29:50,626 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 16:29:50,627 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A3FAA85399704DD1B3C9BA9BEC2DC0AA
2016-01-14 16:29:50,627 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 16:29:50,628 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 16:29:50,628 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788990455_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452788990455_0001_1, initTime=1452788990626
2016-01-14 16:29:50,628 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788990455_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 16:29:50,628 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788990455_0001_1][Event:DAG_STARTED]: dagID=dag_1452788990455_0001_1, startTime=1452788990628
2016-01-14 16:29:50,628 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 16:29:50,629 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 16:29:50,629 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788990455_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 16:29:50,629 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A3FAA85399704DD1B3C9BA9BEC2DC0AA : {91C73DE081BA4AACB0277EA95AC43EC4={InputName=91C73DE081BA4AACB0277EA95AC43EC4}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 16:29:50,629 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 91C73DE081BA4AACB0277EA95AC43EC4, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 16:29:50,629 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452788990455_0001_1_00 [A3FAA85399704DD1B3C9BA9BEC2DC0AA]
2016-01-14 16:29:50,630 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452788990455_0001_1_00 [A3FAA85399704DD1B3C9BA9BEC2DC0AA]
2016-01-14 16:29:50,630 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452788990455_0001_1_00 [A3FAA85399704DD1B3C9BA9BEC2DC0AA]
2016-01-14 16:29:50,630 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452788990455_0001_1_00 [A3FAA85399704DD1B3C9BA9BEC2DC0AA]. Starting root input initializers: 1
2016-01-14 16:29:50,631 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788990455_0001_1_00 [A3FAA85399704DD1B3C9BA9BEC2DC0AA] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 16:29:50,631 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...m/testFilterAll/filterall, applicationId=application_1452788990455_0001, dagName=[75BDD423698248BA91D8AA472D13AC05/E385F60660144B8F9DBAFAA6D72A8978] test/(1/1) ...m/testFilterAll/filterall
2016-01-14 16:29:50,631 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 91C73DE081BA4AACB0277EA95AC43EC4 on vertex vertex_1452788990455_0001_1_00 [A3FAA85399704DD1B3C9BA9BEC2DC0AA]
2016-01-14 16:29:50,633 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 16:29:50,633 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 91C73DE081BA4AACB0277EA95AC43EC4 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 16:29:50,633 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 16:29:50,634 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 16:29:50,637 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-01-14 16:29:50,638 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 91C73DE081BA4AACB0277EA95AC43EC4 on vertex vertex_1452788990455_0001_1_00 [A3FAA85399704DD1B3C9BA9BEC2DC0AA]
2016-01-14 16:29:50,638 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788990455_0001_1_00 [A3FAA85399704DD1B3C9BA9BEC2DC0AA] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:50,638 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {91C73DE081BA4AACB0277EA95AC43EC4=forAllWorkUnits=true, update=[1]}
2016-01-14 16:29:50,638 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452788990455_0001_1_00 [A3FAA85399704DD1B3C9BA9BEC2DC0AA] parallelism set to 1
2016-01-14 16:29:50,638 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788990455_0001_1_00 [A3FAA85399704DD1B3C9BA9BEC2DC0AA] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:50,639 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788990455_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788990455_0001_1_00, eventCount=1
2016-01-14 16:29:50,639 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452788990455_0001_1_00 [A3FAA85399704DD1B3C9BA9BEC2DC0AA]
2016-01-14 16:29:50,639 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=7E2A019959A94B5CB5D8C01C591FEAA2, vertexId=vertex_1452788990455_0001_1_00 [A3FAA85399704DD1B3C9BA9BEC2DC0AA], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 16:29:50,642 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=7E2A019959A94B5CB5D8C01C591FEAA2, vertexId=vertex_1452788990455_0001_1_00 [A3FAA85399704DD1B3C9BA9BEC2DC0AA]
2016-01-14 16:29:50,644 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=7E2A019959A94B5CB5D8C01C591FEAA2, vertexName=A3FAA85399704DD1B3C9BA9BEC2DC0AA, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 16:29:50,644 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=7E2A019959A94B5CB5D8C01C591FEAA2, vertexId=vertex_1452788990455_0001_1_00 [A3FAA85399704DD1B3C9BA9BEC2DC0AA]
2016-01-14 16:29:50,645 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:50,646 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788990455_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A3FAA85399704DD1B3C9BA9BEC2DC0AA, vertexId=vertex_1452788990455_0001_1_00, initRequestedTime=1452788990629, initedTime=1452788990645, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 16:29:50,646 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788990455_0001_1_00 [A3FAA85399704DD1B3C9BA9BEC2DC0AA] with distanceFromRoot: 0
2016-01-14 16:29:50,646 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788990455_0001_1_00 [A3FAA85399704DD1B3C9BA9BEC2DC0AA] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 16:29:50,646 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A3FAA85399704DD1B3C9BA9BEC2DC0AA
2016-01-14 16:29:50,646 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788990455_0001_1_00 [A3FAA85399704DD1B3C9BA9BEC2DC0AA]
2016-01-14 16:29:50,647 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:50,647 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788990455_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788990455_0001_1_00, startRequestedTime=1452788990646, startedTime=1452788990646
2016-01-14 16:29:50,647 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788990455_0001_1_00 [A3FAA85399704DD1B3C9BA9BEC2DC0AA] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:50,648 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:50,648 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788990455_0001_1][Event:TASK_STARTED]: vertexName=A3FAA85399704DD1B3C9BA9BEC2DC0AA, taskId=task_1452788990455_0001_1_00_000000, scheduledTime=1452788990648, launchTime=1452788990648
2016-01-14 16:29:50,649 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788990455_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:50,649 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788990455_0001_1_00 [A3FAA85399704DD1B3C9BA9BEC2DC0AA]
2016-01-14 16:29:50,649 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [75BDD423698248BA91D8AA472D13AC05/E385F60660144B8F9DBAFAA6D72A8978] test/(1/1) ...m/testFilterAll/filterall, VertexName: A3FAA85399704DD1B3C9BA9BEC2DC0AA, VertexParallelism: 1, TaskAttemptID:attempt_1452788990455_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=91C73DE081BA4AACB0277EA95AC43EC4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=7E2A019959A94B5CB5D8C01C591FEAA2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:50,649 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788990455_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:50,649 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:50,650 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 16:29:50,651 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 16:29:50,651 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 16:29:50,652 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788990455_0001_00_000001
2016-01-14 16:29:50,652 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788990455_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:50,654 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788990455_0001_00_000001
2016-01-14 16:29:50,654 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 16:29:50,654 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452788990455_0001, with dag id: dag_1452788990455_0001_1
2016-01-14 16:29:50,661 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788990455_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:50,661 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:50,662 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788990455_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788990455_0001_00_000001, launchTime=1452788990661
2016-01-14 16:29:50,663 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788990455_0001_00_000001
2016-01-14 16:29:50,665 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788990455_0001_1_00_000000_0] to container: [container_1452788990455_0001_00_000001]
2016-01-14 16:29:50,665 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788990455_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:50,666 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788990455_0001_1_00_000000_0] started. Is using containerId: [container_1452788990455_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 16:29:50,667 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:50,667 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788990455_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A3FAA85399704DD1B3C9BA9BEC2DC0AA, taskAttemptId=attempt_1452788990455_0001_1_00_000000_0, startTime=1452788990666, containerId=container_1452788990455_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788990455_0001_00_000001/teamcity, completedLogs=
2016-01-14 16:29:50,667 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788990455_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:50,667 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788990455_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:50,667 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788990455_0001_00_000001 given task: attempt_1452788990455_0001_1_00_000000_0
2016-01-14 16:29:50,668 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788990455_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788990455_0001_1_00_000000_0
2016-01-14 16:29:50 Starting to run new task attempt: attempt_1452788990455_0001_1_00_000000_0
2016-01-14 16:29:50,671 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788990455_0001_1_00_000000_0
2016-01-14 16:29:50,677 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:50,677 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:50,677 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:50,678 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:50,678 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:50,678 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:50,679 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [75BDD423698248BA91D8AA472D13AC05/E385F60660144B8F9DBAFAA6D72A8978] test/(1/1) ...m/testFilterAll/filterall, VertexName: A3FAA85399704DD1B3C9BA9BEC2DC0AA, VertexParallelism: 1, TaskAttemptID:attempt_1452788990455_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=91C73DE081BA4AACB0277EA95AC43EC4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=7E2A019959A94B5CB5D8C01C591FEAA2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:50,679 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:50,681 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788990455_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 16:29:50,681 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788990455_0001_1_00_000000_0
2016-01-14 16:29:50,681 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:50,682 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:50,682 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=91C73DE081BA4AACB0277EA95AC43EC4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 16:29:50,683 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:50,683 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 91C73DE081BA4AACB0277EA95AC43EC4
2016-01-14 16:29:50,683 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 91C73DE081BA4AACB0277EA95AC43EC4
2016-01-14 16:29:50,685 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:50,688 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:50,694 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 16:29:50,694 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 91C73DE081BA4AACB0277EA95AC43EC4
2016-01-14 16:29:50,694 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 91C73DE081BA4AACB0277EA95AC43EC4
2016-01-14 16:29:50,694 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=7E2A019959A94B5CB5D8C01C591FEAA2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 16:29:50,696 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:50,697 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 7E2A019959A94B5CB5D8C01C591FEAA2
2016-01-14 16:29:50,697 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 16:29:50,697 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 7E2A019959A94B5CB5D8C01C591FEAA2
2016-01-14 16:29:50,705 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A3FAA85399704DD1B3C9BA9BEC2DC0AA, ordinal: 0
2016-01-14 16:29:50,708 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A3FAA85399704DD1B3C9BA9BEC2DC0AA, mem on start (mb), free: 267, total: 357, max: 672
2016-01-14 16:29:50,708 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:50,709 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:50,709 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:50,714 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 16:29:50,715 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 7E2A019959A94B5CB5D8C01C591FEAA2
2016-01-14 16:29:50,715 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:50,715 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:50,716 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 16:29:50,716 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:50,716 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 16:29:50,716 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:50,717 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 16:29:50,717 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-01-14 16:29:50,717 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 91C73DE081BA4AACB0277EA95AC43EC4, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 16:29:50,718 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 7E2A019959A94B5CB5D8C01C591FEAA2, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 16:29:50,718 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:50,718 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 91C73DE081BA4AACB0277EA95AC43EC4
2016-01-14 16:29:50,718 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:50,718 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 91C73DE081BA4AACB0277EA95AC43EC4
2016-01-14 16:29:50,719 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 91C73DE081BA4AACB0277EA95AC43EC4 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:50,719 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 16:29:50,719 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:50,720 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:50,720 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788990455_0001_1_00_000000_0
2016-01-14 16:29:50,721 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 16:29:50,722 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@d280526
2016-01-14 16:29:50,723 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 16:29:50,724 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@d280526
2016-01-14 16:29:50,724 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 91C73DE081BA4AACB0277EA95AC43EC4
2016-01-14 16:29:50,724 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall"], id: 7E2A019959A94B5CB5D8C01C591FEAA2
2016-01-14 16:29:50,725 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall"]
2016-01-14 16:29:50,725 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 16:29:50,726 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:50,724 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 16:29:50,727 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 16:29:50,727 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A3FAA85399704DD1B3C9BA9BEC2DC0AA, all 1 inputs ready in: 00:00:00.000
2016-01-14 16:29:50,729 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 16:29:50,729 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 16:29:50,729 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452788990455_0001_1_00_000000_0
2016-01-14 16:29:50,730 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452788990455_0001_1_00_000000_0 given a go for committing the task output.
2016-01-14 16:29:50,730 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788990455_0001_1_00_000000_0
2016-01-14 16:29:50,731 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A3FAA85399704DD1B3C9BA9BEC2DC0AA, mem on close (mb), free: 266, total: 357, max: 672
2016-01-14 16:29:50 Completed running task attempt: attempt_1452788990455_0001_1_00_000000_0
2016-01-14 16:29:50,731 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788990455_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:50,732 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:50,733 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788990455_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A3FAA85399704DD1B3C9BA9BEC2DC0AA, taskAttemptId=attempt_1452788990455_0001_1_00_000000_0, startTime=1452788990666, finishTime=1452788990732, timeTaken=66, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788990727, Process_Duration=3, Process_End_Time=1452788990730, Read_Duration=2, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10
2016-01-14 16:29:50,733 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788990455_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:50,733 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788990455_0001_1_00_000000_0
2016-01-14 16:29:50,733 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788990455_0001_1][Event:TASK_FINISHED]: vertexName=A3FAA85399704DD1B3C9BA9BEC2DC0AA, taskId=task_1452788990455_0001_1_00_000000, startTime=1452788990666, finishTime=1452788990733, timeTaken=67, status=SUCCEEDED, successfulAttemptID=attempt_1452788990455_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788990727, Process_Duration=3, Process_End_Time=1452788990730, Read_Duration=2, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10
2016-01-14 16:29:50,733 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:50,733 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788990455_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:50,734 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788990455_0001_1_00 [A3FAA85399704DD1B3C9BA9BEC2DC0AA] : 1
2016-01-14 16:29:50,734 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788990455_0001_1_00 [A3FAA85399704DD1B3C9BA9BEC2DC0AA]
2016-01-14 16:29:50,734 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:50,735 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:50,736 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788990455_0001_1][Event:VERTEX_FINISHED]: vertexName=A3FAA85399704DD1B3C9BA9BEC2DC0AA, vertexId=vertex_1452788990455_0001_1_00, initRequestedTime=1452788990629, initedTime=1452788990645, startRequestedTime=1452788990646, startedTime=1452788990646, finishTime=1452788990734, timeTaken=88, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788990727, Process_Duration=3, Process_End_Time=1452788990730, Read_Duration=2, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1452788990666, firstTasksToStart=[ task_1452788990455_0001_1_00_000000 ], lastTaskFinishTime=1452788990732, lastTasksToFinish=[ task_1452788990455_0001_1_00_000000 ], minTaskDuration=66, maxTaskDuration=66, avgTaskDuration=66.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788990455_0001_1_00_000000 ], longestDurationTasks=[ task_1452788990455_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:50,736 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788990455_0001_1_00 [A3FAA85399704DD1B3C9BA9BEC2DC0AA] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:50,736 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788990455_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:50,736 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788990455_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788990455_0001_00_000001, stoppedTime=1452788990736, exitStatus=0
2016-01-14 16:29:50,737 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:50,737 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788990455_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:50,737 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788990455_0001_1_00 [A3FAA85399704DD1B3C9BA9BEC2DC0AA] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-01-14 16:29:50,737 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452788990455_0001_1
2016-01-14 16:29:50,737 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788990455_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452788990455_0001_1
2016-01-14 16:29:50,737 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 7E2A019959A94B5CB5D8C01C591FEAA2 for vertex: vertex_1452788990455_0001_1_00 [A3FAA85399704DD1B3C9BA9BEC2DC0AA]
2016-01-14 16:29:50,738 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 19 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1452788990727, Process_Duration=3, Process_End_Time=1452788990730, Read_Duration=2, Tuples_Read=10][cascading.flow.StepCounters Tuples_Read=10]]
2016-01-14 16:29:50,739 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:50,739 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:50,739 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788990455_0001_00_000001
2016-01-14 16:29:50,740 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452788990455_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 16:29:50,740 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788990455_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 16:29:50,740 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452788990455_0001_00_000001, Exiting
2016-01-14 16:29:50,740 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788990455_0001_00_000001
2016-01-14 16:29:50,740 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788990455_0001_00_000001
2016-01-14 16:29:50,743 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788990455_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 16:29:50,744 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452788990455_0001_00_000001 completed successfully
2016-01-14 16:29:50,750 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452788990455_0001_1, queueSize=0
2016-01-14 16:29:50,750 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 16:29:50,750 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788990455_0001_1][Event:DAG_FINISHED]: dagId=dag_1452788990455_0001_1, startTime=1452788990628, finishTime=1452788990749, timeTaken=121, status=SUCCEEDED, diagnostics=, counters=Counters: 21, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788990727, Process_Duration=3, Process_End_Time=1452788990730, Read_Duration=2, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10
2016-01-14 16:29:50,750 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452788990455_0001_1 finished with state: SUCCEEDED
2016-01-14 16:29:50,750 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788990455_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 16:29:50,751 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788990455_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 16:29:50,751 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452788990455_0001_00_000001
2016-01-14 16:29:50 Completed Dag: dag_1452788990455_0001_1
2016-01-14 16:29:50,751 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452788990455_0001_1, dagState=SUCCEEDED
2016-01-14 16:29:50,751 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788990455_0001_1_post
2016-01-14 16:29:50,752 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:50,752 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 16:29:50,758 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...m/testFilterAll/filterall, applicationId=application_1452788990455_0001
2016-01-14 16:29:50,760 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 16:29:50,760 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 16:29:50,760 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 16:29:50,760 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 16:29:50,760 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 16:29:50,761 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 16:29:50,762 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 16:29:50,762 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 16:29:50,762 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 16:29:50,762 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 16:29:50,763 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 16:29:50,763 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 16:29:50,763 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 16:29:50,767 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/75BDD423698248BA91D8AA472D13AC05/E385F60660144B8F9DBAFAA6D72A8978/.tez/application_1452788990455_0001
2016-01-14 16:29:50,768 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:00.332
2016-01-14 16:29:50,804 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testFilterAll/filterall
2016-01-14 16:29:50,834 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 16:29:50,834 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 16:29:50,878 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-01-14 16:29:50,899 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.035
2016-01-14 16:29:50,899 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 16:29:50,900 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 16:29:50,900 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 16:29:50,924 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt
2016-01-14 16:29:50,934 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown
2016-01-14 16:29:50,987 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 16:29:50,988 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 16:29:50,988 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown"]
2016-01-14 16:29:50,988 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 16:29:50,989 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 16:29:50,989 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 16:29:50,990 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...own/simplerelativeunknown
2016-01-14 16:29:50,990 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 16:29:51,003 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 16:29:51,008 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:51,008 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 16:29:51,008 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 16:29:51,018 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/99DA2911845047BD9B8ED54BD9291791/D3D213334EB04CC0A05F061DE5E00ED5/.tez/application_1452788991008_0001 doesn't exist and is created
2016-01-14 16:29:51,039 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 16:29:51,039 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 16:29:51,040 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/99DA2911845047BD9B8ED54BD9291791/D3D213334EB04CC0A05F061DE5E00ED5/.tez/application_1452788991008_0001
2016-01-14 16:29:51,044 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452788991008_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:51,054 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 16:29:51,055 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 16:29:51,057 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 16:29:51,057 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 16:29:51,057 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/99DA2911845047BD9B8ED54BD9291791/D3D213334EB04CC0A05F061DE5E00ED5/.tez/application_1452788991008_0001/localmode-log-dir
2016-01-14 16:29:51,058 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/99DA2911845047BD9B8ED54BD9291791/D3D213334EB04CC0A05F061DE5E00ED5/.tez/application_1452788991008_0001/localmode-log-dir/history.txt.appattempt_1452788991008_0001_000000, maxErrors=10
2016-01-14 16:29:51,058 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 16:29:51,058 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452788991008_0001_000000, appSubmitTime=1452788991043, launchTime=1452788991043
2016-01-14 16:29:51,063 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 16:29:51,066 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 16:29:51,067 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 16:29:51,069 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:56975
2016-01-14 16:29:51,069 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 16:29:51,073 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 16:29:51,075 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452788991008_0001_000000, startTime=1452788991075
2016-01-14 16:29:51,075 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 16:29:51,075 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 16:29:51,140 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 16:29:51,140 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 16:29:51,140 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...own/simplerelativeunknown, applicationId=application_1452788991008_0001, dagName=[99DA2911845047BD9B8ED54BD9291791/D3D213334EB04CC0A05F061DE5E00ED5] test/(1/1) ...own/simplerelativeunknown
2016-01-14 16:29:51,157 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [99DA2911845047BD9B8ED54BD9291791/D3D213334EB04CC0A05F061DE5E00ED5] test/(1/1) ...own/simplerelativeunknown
2016-01-14 16:29:51,158 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452788991008_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/99DA2911845047BD9B8ED54BD9291791/D3D213334EB04CC0A05F061DE5E00ED5/.tez/application_1452788991008_0001/localmode-log-dir/dag_1452788991008_0001_1.dot
2016-01-14 16:29:51,159 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/99DA2911845047BD9B8ED54BD9291791/D3D213334EB04CC0A05F061DE5E00ED5/.tez/application_1452788991008_0001/localmode-log-dir/dag_1452788991008_0001_1-tez-dag.pb.txt
2016-01-14 16:29:51,172 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788991008_0001_1
2016-01-14 16:29:51,172 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:51,172 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [99DA2911845047BD9B8ED54BD9291791/D3D213334EB04CC0A05F061DE5E00ED5] test/(1/1) ...own/simplerelativeunknown
2016-01-14 16:29:51 Running Dag: dag_1452788991008_0001_1
2016-01-14 16:29:51,185 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991008_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452788991008_0001_1, submitTime=1452788991157
2016-01-14 16:29:51,185 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 16:29:51,186 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E66065F328C24D4BAC0EA9A30C5B5505
2016-01-14 16:29:51,186 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 16:29:51,187 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991008_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452788991008_0001_1, initTime=1452788991185
2016-01-14 16:29:51,187 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 16:29:51,187 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788991008_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 16:29:51,187 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991008_0001_1][Event:DAG_STARTED]: dagID=dag_1452788991008_0001_1, startTime=1452788991187
2016-01-14 16:29:51,188 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 16:29:51,187 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 16:29:51,188 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788991008_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 16:29:51,188 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C4CBFBB69A884E1B9AB62519EE257082 : {F6C490CA6A3C4520886364A2AE7692FC={InputName=F6C490CA6A3C4520886364A2AE7692FC}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 16:29:51,188 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: F6C490CA6A3C4520886364A2AE7692FC, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 16:29:51,188 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452788991008_0001_1_00 [C4CBFBB69A884E1B9AB62519EE257082]
2016-01-14 16:29:51,188 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452788991008_0001_1_00 [C4CBFBB69A884E1B9AB62519EE257082]
2016-01-14 16:29:51,188 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452788991008_0001_1_00 [C4CBFBB69A884E1B9AB62519EE257082]
2016-01-14 16:29:51,189 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452788991008_0001_1_00 [C4CBFBB69A884E1B9AB62519EE257082]. Starting root input initializers: 1
2016-01-14 16:29:51,189 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788991008_0001_1_00 [C4CBFBB69A884E1B9AB62519EE257082] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 16:29:51,189 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452788991008_0001_1_01 [E66065F328C24D4BAC0EA9A30C5B5505]
2016-01-14 16:29:51,190 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: F6C490CA6A3C4520886364A2AE7692FC on vertex vertex_1452788991008_0001_1_00 [C4CBFBB69A884E1B9AB62519EE257082]
2016-01-14 16:29:51,191 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...own/simplerelativeunknown, applicationId=application_1452788991008_0001, dagName=[99DA2911845047BD9B8ED54BD9291791/D3D213334EB04CC0A05F061DE5E00ED5] test/(1/1) ...own/simplerelativeunknown
2016-01-14 16:29:51,192 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 16:29:51,192 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 16:29:51,193 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input F6C490CA6A3C4520886364A2AE7692FC asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 16:29:51,193 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452788991008_0001_1_01 [E66065F328C24D4BAC0EA9A30C5B5505]
2016-01-14 16:29:51,193 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452788991008_0001_1_01 [E66065F328C24D4BAC0EA9A30C5B5505]
2016-01-14 16:29:51,193 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 16:29:51,193 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452788991008_0001_1_01 [E66065F328C24D4BAC0EA9A30C5B5505]
2016-01-14 16:29:51,193 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 16:29:51,194 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=71D6D8ED3E7949A7BFF4D30517050AF3, vertexId=vertex_1452788991008_0001_1_01 [E66065F328C24D4BAC0EA9A30C5B5505], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 16:29:51,194 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=71D6D8ED3E7949A7BFF4D30517050AF3, vertexId=vertex_1452788991008_0001_1_01 [E66065F328C24D4BAC0EA9A30C5B5505]
2016-01-14 16:29:51,196 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=71D6D8ED3E7949A7BFF4D30517050AF3, vertexName=E66065F328C24D4BAC0EA9A30C5B5505, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 16:29:51,196 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=71D6D8ED3E7949A7BFF4D30517050AF3, vertexId=vertex_1452788991008_0001_1_01 [E66065F328C24D4BAC0EA9A30C5B5505]
2016-01-14 16:29:51,197 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-01-14 16:29:51,197 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: F6C490CA6A3C4520886364A2AE7692FC on vertex vertex_1452788991008_0001_1_00 [C4CBFBB69A884E1B9AB62519EE257082]
2016-01-14 16:29:51,198 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:51,198 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991008_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E66065F328C24D4BAC0EA9A30C5B5505, vertexId=vertex_1452788991008_0001_1_01, initRequestedTime=1452788991189, initedTime=1452788991198, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 16:29:51,198 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788991008_0001_1_01 [E66065F328C24D4BAC0EA9A30C5B5505] transitioned from NEW to INITED due to event V_INIT
2016-01-14 16:29:51,198 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788991008_0001_1_00 [C4CBFBB69A884E1B9AB62519EE257082] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:51,199 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {F6C490CA6A3C4520886364A2AE7692FC=forAllWorkUnits=true, update=[1]}
2016-01-14 16:29:51,199 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452788991008_0001_1_00 [C4CBFBB69A884E1B9AB62519EE257082] parallelism set to 1
2016-01-14 16:29:51,199 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788991008_0001_1_00 [C4CBFBB69A884E1B9AB62519EE257082] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:51,200 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991008_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788991008_0001_1_00, eventCount=1
2016-01-14 16:29:51,200 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:51,200 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991008_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C4CBFBB69A884E1B9AB62519EE257082, vertexId=vertex_1452788991008_0001_1_00, initRequestedTime=1452788991188, initedTime=1452788991200, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 16:29:51,201 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788991008_0001_1_00 [C4CBFBB69A884E1B9AB62519EE257082] with distanceFromRoot: 0
2016-01-14 16:29:51,201 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788991008_0001_1_00 [C4CBFBB69A884E1B9AB62519EE257082] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 16:29:51,201 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C4CBFBB69A884E1B9AB62519EE257082
2016-01-14 16:29:51,202 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788991008_0001_1_00 [C4CBFBB69A884E1B9AB62519EE257082]
2016-01-14 16:29:51,202 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:51,202 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991008_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788991008_0001_1_00, startRequestedTime=1452788991201, startedTime=1452788991201
2016-01-14 16:29:51,203 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788991008_0001_1_00 [C4CBFBB69A884E1B9AB62519EE257082] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:51,203 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:51,203 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991008_0001_1][Event:TASK_STARTED]: vertexName=C4CBFBB69A884E1B9AB62519EE257082, taskId=task_1452788991008_0001_1_00_000000, scheduledTime=1452788991203, launchTime=1452788991203
2016-01-14 16:29:51,204 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788991008_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:51,204 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452788991008_0001_1_00 for vertex: vertex_1452788991008_0001_1_01 [E66065F328C24D4BAC0EA9A30C5B5505] numStartedSources: 1 numSources: 1
2016-01-14 16:29:51,204 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788991008_0001_1_01 [E66065F328C24D4BAC0EA9A30C5B5505] with distanceFromRoot: 1
2016-01-14 16:29:51,205 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788991008_0001_1_00 [C4CBFBB69A884E1B9AB62519EE257082]
2016-01-14 16:29:51,205 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C4CBFBB69A884E1B9AB62519EE257082 in vertex: E66065F328C24D4BAC0EA9A30C5B5505
2016-01-14 16:29:51,205 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E66065F328C24D4BAC0EA9A30C5B5505 with 1 source tasks and 1 pending tasks
2016-01-14 16:29:51,206 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:51,206 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991008_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788991008_0001_1_01, startRequestedTime=1452788991205, startedTime=1452788991205
2016-01-14 16:29:51,206 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788991008_0001_1_01 [E66065F328C24D4BAC0EA9A30C5B5505] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:51,207 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [99DA2911845047BD9B8ED54BD9291791/D3D213334EB04CC0A05F061DE5E00ED5] test/(1/1) ...own/simplerelativeunknown, VertexName: C4CBFBB69A884E1B9AB62519EE257082, VertexParallelism: 1, TaskAttemptID:attempt_1452788991008_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F6C490CA6A3C4520886364A2AE7692FC, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E66065F328C24D4BAC0EA9A30C5B5505, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 16:29:51,207 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788991008_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:51,207 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:51,209 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 16:29:51,210 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 16:29:51,210 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 16:29:51,210 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788991008_0001_00_000001
2016-01-14 16:29:51,210 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991008_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:51,212 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452788991008_0001, with dag id: dag_1452788991008_0001_1
2016-01-14 16:29:51,213 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788991008_0001_00_000001
2016-01-14 16:29:51,213 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 16:29:51,214 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991008_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:51,214 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:51,215 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991008_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788991008_0001_00_000001, launchTime=1452788991214
2016-01-14 16:29:51,215 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788991008_0001_00_000001
2016-01-14 16:29:51,216 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788991008_0001_1_00_000000_0] to container: [container_1452788991008_0001_00_000001]
2016-01-14 16:29:51,217 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991008_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:51,217 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788991008_0001_1_00_000000_0] started. Is using containerId: [container_1452788991008_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 16:29:51,218 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991008_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C4CBFBB69A884E1B9AB62519EE257082, taskAttemptId=attempt_1452788991008_0001_1_00_000000_0, startTime=1452788991217, containerId=container_1452788991008_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788991008_0001_00_000001/teamcity, completedLogs=
2016-01-14 16:29:51,218 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788991008_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:51,218 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:51,218 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788991008_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:51,219 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788991008_0001_00_000001 given task: attempt_1452788991008_0001_1_00_000000_0
2016-01-14 16:29:51,219 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788991008_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788991008_0001_1_00_000000_0
2016-01-14 16:29:51 Starting to run new task attempt: attempt_1452788991008_0001_1_00_000000_0
2016-01-14 16:29:51,220 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788991008_0001_1_00_000000_0
2016-01-14 16:29:51,220 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:51,220 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:51,220 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:51,221 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:51,221 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:51,221 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:51,221 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [99DA2911845047BD9B8ED54BD9291791/D3D213334EB04CC0A05F061DE5E00ED5] test/(1/1) ...own/simplerelativeunknown, VertexName: C4CBFBB69A884E1B9AB62519EE257082, VertexParallelism: 1, TaskAttemptID:attempt_1452788991008_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F6C490CA6A3C4520886364A2AE7692FC, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E66065F328C24D4BAC0EA9A30C5B5505, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 16:29:51,222 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:51,222 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788991008_0001_1_00_000000_0
2016-01-14 16:29:51,223 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:51,223 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788991008_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 16:29:51,223 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:51,224 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F6C490CA6A3C4520886364A2AE7692FC, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 16:29:51,224 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E66065F328C24D4BAC0EA9A30C5B5505, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 16:29:51,224 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:51,225 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:51,225 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:51,225 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E66065F328C24D4BAC0EA9A30C5B5505
2016-01-14 16:29:51,225 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F6C490CA6A3C4520886364A2AE7692FC
2016-01-14 16:29:51,226 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: F6C490CA6A3C4520886364A2AE7692FC
2016-01-14 16:29:51,227 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 16:29:51,227 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E66065F328C24D4BAC0EA9A30C5B5505
2016-01-14 16:29:51,227 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E66065F328C24D4BAC0EA9A30C5B5505
2016-01-14 16:29:51,228 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 16:29:51,228 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: F6C490CA6A3C4520886364A2AE7692FC
2016-01-14 16:29:51,228 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F6C490CA6A3C4520886364A2AE7692FC
2016-01-14 16:29:51,229 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:51,238 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C4CBFBB69A884E1B9AB62519EE257082, ordinal: 0
2016-01-14 16:29:51,238 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C4CBFBB69A884E1B9AB62519EE257082, mem on start (mb), free: 188, total: 357, max: 672
2016-01-14 16:29:51,238 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:51,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:51,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:51,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:51,239 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:51,240 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 16:29:51,240 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:51,241 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 16:29:51,241 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 16:29:51,241 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 16:29:51,241 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F6C490CA6A3C4520886364A2AE7692FC, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 16:29:51,242 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E66065F328C24D4BAC0EA9A30C5B5505, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 16:29:51,242 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:51,242 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F6C490CA6A3C4520886364A2AE7692FC being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:51,242 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F6C490CA6A3C4520886364A2AE7692FC
2016-01-14 16:29:51,243 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:51,243 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F6C490CA6A3C4520886364A2AE7692FC
2016-01-14 16:29:51,242 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 16:29:51,244 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:51,244 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:51,245 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788991008_0001_1_00_000000_0
2016-01-14 16:29:51,245 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 16:29:51,245 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7a9a6344
2016-01-14 16:29:51,246 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7a9a6344
2016-01-14 16:29:51,246 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 16:29:51,246 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 16:29:51,249 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 16:29:51,249 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"] streamed: true, id: F6C490CA6A3C4520886364A2AE7692FC
2016-01-14 16:29:51,250 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:-1]], id: 4A3F8E1BF8D24B4791A11C776EABD5CE
2016-01-14 16:29:51,250 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:-1]] 4A3F8E1BF8D24B4791A11C776EABD5CE
2016-01-14 16:29:51,251 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@53761fdb; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@70258dab; comparator=cascading.tuple.hadoop.util.TupleComparator@31780f12; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 16:29:51,251 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 16:29:51,252 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 16:29:51,252 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 16:29:51,274 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 16:29:51,274 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 16:29:51,274 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 16:29:51,275 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 16:29:51,275 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 16:29:51,276 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 16:29:51,276 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:51,276 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C4CBFBB69A884E1B9AB62519EE257082, all 1 inputs ready in: 00:00:00.000
2016-01-14 16:29:51,279 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788991008_0001_1_00_000000_0
2016-01-14 16:29:51,279 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C4CBFBB69A884E1B9AB62519EE257082, mem on close (mb), free: 226, total: 355, max: 672
2016-01-14 16:29:51,279 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 16:29:51,280 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 16:29:51,280 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-01-14 16:29:51,280 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-01-14 16:29:51,283 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 16:29:51 Completed running task attempt: attempt_1452788991008_0001_1_00_000000_0
2016-01-14 16:29:51,286 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788991008_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:51,287 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=372768768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452788991277, Process_Duration=2, Process_End_Time=1452788991279, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-01-14 16:29:51,287 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991008_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788991008_0001_1_00, eventCount=1
2016-01-14 16:29:51,287 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:51,288 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:51,288 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788991008_0001_00_000001
2016-01-14 16:29:51,288 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991008_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C4CBFBB69A884E1B9AB62519EE257082, taskAttemptId=attempt_1452788991008_0001_1_00_000000_0, startTime=1452788991217, finishTime=1452788991288, timeTaken=71, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=372768768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788991277, Process_Duration=2, Process_End_Time=1452788991279, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 16:29:51,288 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788991008_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:51,288 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:51,288 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:51,288 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788991008_0001_1_00_000000_0
2016-01-14 16:29:51,289 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:51,290 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991008_0001_1][Event:TASK_FINISHED]: vertexName=C4CBFBB69A884E1B9AB62519EE257082, taskId=task_1452788991008_0001_1_00_000000, startTime=1452788991217, finishTime=1452788991289, timeTaken=72, status=SUCCEEDED, successfulAttemptID=attempt_1452788991008_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=372768768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788991277, Process_Duration=2, Process_End_Time=1452788991279, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 16:29:51,290 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788991008_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:51,290 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788991008_0001_1_00 [C4CBFBB69A884E1B9AB62519EE257082] : 1
2016-01-14 16:29:51,290 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788991008_0001_1_00 [C4CBFBB69A884E1B9AB62519EE257082]
2016-01-14 16:29:51,291 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:51,292 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991008_0001_1][Event:VERTEX_FINISHED]: vertexName=C4CBFBB69A884E1B9AB62519EE257082, vertexId=vertex_1452788991008_0001_1_00, initRequestedTime=1452788991188, initedTime=1452788991200, startRequestedTime=1452788991201, startedTime=1452788991201, finishTime=1452788991290, timeTaken=89, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=372768768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788991277, Process_Duration=2, Process_End_Time=1452788991279, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1452788991217, firstTasksToStart=[ task_1452788991008_0001_1_00_000000 ], lastTaskFinishTime=1452788991288, lastTasksToFinish=[ task_1452788991008_0001_1_00_000000 ], minTaskDuration=71, maxTaskDuration=71, avgTaskDuration=71.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788991008_0001_1_00_000000 ], longestDurationTasks=[ task_1452788991008_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:51,292 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788991008_0001_1_00 [C4CBFBB69A884E1B9AB62519EE257082] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:51,292 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991008_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:51,293 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452788991008_0001_1_01 [E66065F328C24D4BAC0EA9A30C5B5505] attempt: attempt_1452788991008_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 16:29:51,293 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E66065F328C24D4BAC0EA9A30C5B5505
2016-01-14 16:29:51,293 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788991008_0001_1_01 [E66065F328C24D4BAC0EA9A30C5B5505]
2016-01-14 16:29:51,294 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991008_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788991008_0001_00_000001, stoppedTime=1452788991294, exitStatus=0
2016-01-14 16:29:51,294 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991008_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:51,294 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788991008_0001_1_00 [C4CBFBB69A884E1B9AB62519EE257082] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 16:29:51,294 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:51,295 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:51,296 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991008_0001_1][Event:TASK_STARTED]: vertexName=E66065F328C24D4BAC0EA9A30C5B5505, taskId=task_1452788991008_0001_1_01_000000, scheduledTime=1452788991295, launchTime=1452788991295
2016-01-14 16:29:51,296 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788991008_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:51,296 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788991008_0001_1_01 [E66065F328C24D4BAC0EA9A30C5B5505]
2016-01-14 16:29:51,296 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452788991008_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 16:29:51,296 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [99DA2911845047BD9B8ED54BD9291791/D3D213334EB04CC0A05F061DE5E00ED5] test/(1/1) ...own/simplerelativeunknown, VertexName: E66065F328C24D4BAC0EA9A30C5B5505, VertexParallelism: 1, TaskAttemptID:attempt_1452788991008_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C4CBFBB69A884E1B9AB62519EE257082, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=71D6D8ED3E7949A7BFF4D30517050AF3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:51,297 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788991008_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:51,297 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991008_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 16:29:51,297 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:51,298 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788991008_0001_00_000002
2016-01-14 16:29:51,298 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991008_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:51,299 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788991008_0001_00_000002
2016-01-14 16:29:51,299 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 16:29:51,300 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452788991008_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 16:29:51,300 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991008_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:51,300 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:51,300 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788991008_0001_00_000001: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 16:29:51,301 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452788991008_0001_00_000001, Exiting
2016-01-14 16:29:51,301 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788991008_0001_00_000001
2016-01-14 16:29:51,301 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788991008_0001_00_000001
2016-01-14 16:29:51,301 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991008_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788991008_0001_00_000002, launchTime=1452788991300
2016-01-14 16:29:51,304 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788991008_0001_00_000002
2016-01-14 16:29:51,304 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788991008_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 16:29:51,304 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452788991008_0001_00_000001 completed successfully
2016-01-14 16:29:51,304 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788991008_0001_1_01_000000_0] to container: [container_1452788991008_0001_00_000002]
2016-01-14 16:29:51,305 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991008_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:51,305 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991008_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 16:29:51,306 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788991008_0001_1_01_000000_0] started. Is using containerId: [container_1452788991008_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 16:29:51,306 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991008_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E66065F328C24D4BAC0EA9A30C5B5505, taskAttemptId=attempt_1452788991008_0001_1_01_000000_0, startTime=1452788991305, containerId=container_1452788991008_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788991008_0001_00_000002/teamcity, completedLogs=
2016-01-14 16:29:51,306 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788991008_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:51,306 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788991008_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:51,307 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:51,307 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788991008_0001_00_000002 given task: attempt_1452788991008_0001_1_01_000000_0
2016-01-14 16:29:51,307 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788991008_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788991008_0001_1_01_000000_0
2016-01-14 16:29:51 Starting to run new task attempt: attempt_1452788991008_0001_1_01_000000_0
2016-01-14 16:29:51,308 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788991008_0001_1_01_000000_0
2016-01-14 16:29:51,308 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:51,308 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:51,308 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:51,308 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:51,309 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:51,309 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:51,309 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [99DA2911845047BD9B8ED54BD9291791/D3D213334EB04CC0A05F061DE5E00ED5] test/(1/1) ...own/simplerelativeunknown, VertexName: E66065F328C24D4BAC0EA9A30C5B5505, VertexParallelism: 1, TaskAttemptID:attempt_1452788991008_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C4CBFBB69A884E1B9AB62519EE257082, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=71D6D8ED3E7949A7BFF4D30517050AF3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:51,309 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:51,309 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788991008_0001_1_01_000000_0
2016-01-14 16:29:51,310 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:51,310 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:51,311 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788991008_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 16:29:51,311 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:51,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=71D6D8ED3E7949A7BFF4D30517050AF3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 16:29:51,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:51,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 71D6D8ED3E7949A7BFF4D30517050AF3
2016-01-14 16:29:51,313 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 16:29:51,313 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 71D6D8ED3E7949A7BFF4D30517050AF3
2016-01-14 16:29:51,314 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:51,322 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C4CBFBB69A884E1B9AB62519EE257082, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 16:29:51,323 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:51,323 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C4CBFBB69A884E1B9AB62519EE257082
2016-01-14 16:29:51,325 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 16:29:51,325 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 16:29:51,326 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C4CBFBB69A884E1B9AB62519EE257082
2016-01-14 16:29:51,328 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C4CBFBB69A884E1B9AB62519EE257082
2016-01-14 16:29:51,335 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E66065F328C24D4BAC0EA9A30C5B5505, ordinal: 1
2016-01-14 16:29:51,335 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E66065F328C24D4BAC0EA9A30C5B5505, mem on start (mb), free: 220, total: 355, max: 672
2016-01-14 16:29:51,336 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:51,336 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:51,336 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:51,338 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 16:29:51,338 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 71D6D8ED3E7949A7BFF4D30517050AF3
2016-01-14 16:29:51,338 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:51,338 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:51,338 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 16:29:51,339 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:51,339 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 16:29:51,339 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 16:29:51,339 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 16:29:51,339 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 71D6D8ED3E7949A7BFF4D30517050AF3, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 16:29:51,339 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C4CBFBB69A884E1B9AB62519EE257082, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 16:29:51,339 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:51,340 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C4CBFBB69A884E1B9AB62519EE257082
2016-01-14 16:29:51,340 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C4CBFBB69A884E1B9AB62519EE257082 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:51,340 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 16:29:51,341 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 16:29:51,341 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C4CBFBB69A884E1B9AB62519EE257082 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 16:29:51,341 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 16:29:51,341 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 16:29:51,342 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 16:29:51,342 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C4CBFBB69A884E1B9AB62519EE257082
2016-01-14 16:29:51,342 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 16:29:51,343 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:51,343 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:51,343 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788991008_0001_1_01_000000_0
2016-01-14 16:29:51,344 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452788991008_0001_1_00_000000_0_10049, runDuration: 59279, ]
2016-01-14 16:29:51,346 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452788991008_0001_1_00_000000_0_10049] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1452788991346, TimeTaken=1, Rate=0.09 MB/s
2016-01-14 16:29:51,346 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C4CBFBB69A884E1B9AB62519EE257082
2016-01-14 16:29:51,346 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 16:29:51,346 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:-1]]
2016-01-14 16:29:51,347 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:-1]] streamed: true, id: 4A3F8E1BF8D24B4791A11C776EABD5CE
2016-01-14 16:29:51,348 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown"], id: 71D6D8ED3E7949A7BFF4D30517050AF3
2016-01-14 16:29:51,346 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [C4CBFBB69A884E1B9AB62519EE257082] #8 in 2ms
2016-01-14 16:29:51,349 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown"]
2016-01-14 16:29:51,349 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C4CBFBB69A884E1B9AB62519EE257082] #8, status:false, isInterrupted:false
2016-01-14 16:29:51,349 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:-1]] 4A3F8E1BF8D24B4791A11C776EABD5CE, for 1 inputs
2016-01-14 16:29:51,350 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 16:29:51,350 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-01-14 16:29:51,350 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 16:29:51,350 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 16:29:51,351 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-01-14 16:29:51,351 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C4CBFBB69A884E1B9AB62519EE257082
2016-01-14 16:29:51,351 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 16:29:51,351 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E66065F328C24D4BAC0EA9A30C5B5505, all 1 inputs ready in: 00:00:00.001
2016-01-14 16:29:51,352 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@2e50bfef; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3768fbaa
2016-01-14 16:29:51,353 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 16:29:51,353 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 16:29:51,354 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452788991008_0001_1_01_000000_0
2016-01-14 16:29:51,354 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452788991008_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 16:29:51,355 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788991008_0001_1_01_000000_0
2016-01-14 16:29:51,355 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E66065F328C24D4BAC0EA9A30C5B5505, mem on close (mb), free: 218, total: 355, max: 672
2016-01-14 16:29:51,355 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C4CBFBB69A884E1B9AB62519EE257082
2016-01-14 16:29:51,355 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 16:29:51,355 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 16:29:51,356 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 16:29:51,356 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 16:29:51,356 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 16:29:51 Completed running task attempt: attempt_1452788991008_0001_1_01_000000_0
2016-01-14 16:29:51,357 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788991008_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:51,359 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991008_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E66065F328C24D4BAC0EA9A30C5B5505, taskAttemptId=attempt_1452788991008_0001_1_01_000000_0, startTime=1452788991305, finishTime=1452788991359, timeTaken=54, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=127, FILE_BYTES_WRITTEN=32, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=372768768, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788991351, Process_Duration=4, Process_End_Time=1452788991355, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-01-14 16:29:51,359 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788991008_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:51,360 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:51,360 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:51,360 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788991008_0001_1_01_000000_0
2016-01-14 16:29:51,360 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:51,360 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991008_0001_1][Event:TASK_FINISHED]: vertexName=E66065F328C24D4BAC0EA9A30C5B5505, taskId=task_1452788991008_0001_1_01_000000, startTime=1452788991305, finishTime=1452788991360, timeTaken=55, status=SUCCEEDED, successfulAttemptID=attempt_1452788991008_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=127, FILE_BYTES_WRITTEN=32, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=372768768, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788991351, Process_Duration=4, Process_End_Time=1452788991355, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-01-14 16:29:51,361 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788991008_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:51,361 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788991008_0001_1_01 [E66065F328C24D4BAC0EA9A30C5B5505] : 1
2016-01-14 16:29:51,361 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788991008_0001_1_01 [E66065F328C24D4BAC0EA9A30C5B5505]
2016-01-14 16:29:51,361 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=127, FILE_BYTES_WRITTEN=32, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=372768768, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452788991351, Process_Duration=4, Process_End_Time=1452788991355, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-01-14 16:29:51,361 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:51,361 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:51,362 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788991008_0001_00_000002
2016-01-14 16:29:51,362 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991008_0001_1][Event:VERTEX_FINISHED]: vertexName=E66065F328C24D4BAC0EA9A30C5B5505, vertexId=vertex_1452788991008_0001_1_01, initRequestedTime=1452788991189, initedTime=1452788991198, startRequestedTime=1452788991205, startedTime=1452788991205, finishTime=1452788991361, timeTaken=156, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=127, FILE_BYTES_WRITTEN=32, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=372768768, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788991351, Process_Duration=4, Process_End_Time=1452788991355, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1452788991305, firstTasksToStart=[ task_1452788991008_0001_1_01_000000 ], lastTaskFinishTime=1452788991359, lastTasksToFinish=[ task_1452788991008_0001_1_01_000000 ], minTaskDuration=54, maxTaskDuration=54, avgTaskDuration=54.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788991008_0001_1_01_000000 ], longestDurationTasks=[ task_1452788991008_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:51,362 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788991008_0001_1_01 [E66065F328C24D4BAC0EA9A30C5B5505] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:51,362 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:51,362 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991008_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:51,362 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991008_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788991008_0001_00_000002, stoppedTime=1452788991362, exitStatus=0
2016-01-14 16:29:51,362 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991008_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:51,362 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788991008_0001_1_01 [E66065F328C24D4BAC0EA9A30C5B5505] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 16:29:51,363 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452788991008_0001_1
2016-01-14 16:29:51,363 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991008_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452788991008_0001_1
2016-01-14 16:29:51,363 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:51,363 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452788991008_0001_1_00 [C4CBFBB69A884E1B9AB62519EE257082]
2016-01-14 16:29:51,364 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 71D6D8ED3E7949A7BFF4D30517050AF3 for vertex: vertex_1452788991008_0001_1_01 [E66065F328C24D4BAC0EA9A30C5B5505]
2016-01-14 16:29:51,368 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 16:29:51,369 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 16:29:51,372 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452788991008_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 16:29:51,372 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788991008_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 16:29:51,372 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452788991008_0001_00_000002, Exiting
2016-01-14 16:29:51,372 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788991008_0001_00_000002
2016-01-14 16:29:51,372 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788991008_0001_00_000002
2016-01-14 16:29:51,373 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788991008_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 16:29:51,373 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452788991008_0001_00_000002 completed successfully
2016-01-14 16:29:51,376 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452788991008_0001_1, queueSize=0
2016-01-14 16:29:51,376 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 16:29:51,377 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991008_0001_1][Event:DAG_FINISHED]: dagId=dag_1452788991008_0001_1, startTime=1452788991187, finishTime=1452788991374, timeTaken=187, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=189, FILE_BYTES_WRITTEN=159, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=745537536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905577982628, Process_Duration=6, Process_End_Time=2905577982634, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-01-14 16:29:51,377 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452788991008_0001_1 finished with state: SUCCEEDED
2016-01-14 16:29:51,377 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788991008_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 16:29:51,378 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452788991008_0001_00_000002
2016-01-14 16:29:51,378 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991008_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 16:29:51 Completed Dag: dag_1452788991008_0001_1
2016-01-14 16:29:51,379 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452788991008_0001_1, dagState=SUCCEEDED
2016-01-14 16:29:51,379 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788991008_0001_1_post
2016-01-14 16:29:51,379 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:51,379 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 16:29:51,384 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...own/simplerelativeunknown, applicationId=application_1452788991008_0001
2016-01-14 16:29:51,386 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 16:29:51,386 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 16:29:51,386 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 16:29:51,386 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 16:29:51,386 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 16:29:51,387 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 16:29:51,387 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 16:29:51,387 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 16:29:51,387 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 16:29:51,388 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 16:29:51,388 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 16:29:51,388 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 16:29:51,388 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 16:29:51,393 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/99DA2911845047BD9B8ED54BD9291791/D3D213334EB04CC0A05F061DE5E00ED5/.tez/application_1452788991008_0001
2016-01-14 16:29:51,394 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:00.407
2016-01-14 16:29:51,434 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown
2016-01-14 16:29:51,458 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 16:29:51,458 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 16:29:51,511 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.021
2016-01-14 16:29:51,529 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.040
2016-01-14 16:29:51,530 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-01-14 16:29:51,530 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-01-14 16:29:51,530 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 16:29:51,545 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt
2016-01-14 16:29:51,552 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt
2016-01-14 16:29:51,559 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup
2016-01-14 16:29:51,647 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] starting
2016-01-14 16:29:51,647 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-01-14 16:29:51,647 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 16:29:51,647 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup"]
2016-01-14 16:29:51,648 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] parallel execution of steps is enabled: false
2016-01-14 16:29:51,648 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] executing total steps: 1
2016-01-14 16:29:51,648 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] allocating management threads: 1
2016-01-14 16:29:51,648 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] starting step: (1/1) ...estCoGroup/complexcogroup
2016-01-14 16:29:51,648 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [lower*upper] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 16:29:51,661 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] tez session mode enabled: true
2016-01-14 16:29:51,665 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:51,665 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 16:29:51,665 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 16:29:51,674 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/688B204A643045FAA6480444CE09144E/C2CB6F3D18104B02B63CC3351D3E2252/.tez/application_1452788991665_0001 doesn't exist and is created
2016-01-14 16:29:51,690 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 16:29:51,690 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 16:29:51,691 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/688B204A643045FAA6480444CE09144E/C2CB6F3D18104B02B63CC3351D3E2252/.tez/application_1452788991665_0001
2016-01-14 16:29:51,694 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452788991665_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:51,705 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 16:29:51,705 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 16:29:51,707 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 16:29:51,707 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 16:29:51,707 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/688B204A643045FAA6480444CE09144E/C2CB6F3D18104B02B63CC3351D3E2252/.tez/application_1452788991665_0001/localmode-log-dir
2016-01-14 16:29:51,708 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/688B204A643045FAA6480444CE09144E/C2CB6F3D18104B02B63CC3351D3E2252/.tez/application_1452788991665_0001/localmode-log-dir/history.txt.appattempt_1452788991665_0001_000000, maxErrors=10
2016-01-14 16:29:51,708 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 16:29:51,708 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452788991665_0001_000000, appSubmitTime=1452788991693, launchTime=1452788991694
2016-01-14 16:29:51,711 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 16:29:51,712 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 16:29:51,717 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 16:29:51,718 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:45708
2016-01-14 16:29:51,718 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 16:29:51,720 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 16:29:51,721 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452788991665_0001_000000, startTime=1452788991721
2016-01-14 16:29:51,721 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 16:29:51,721 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 16:29:51,791 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 16:29:51,791 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 16:29:51,791 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...estCoGroup/complexcogroup, applicationId=application_1452788991665_0001, dagName=[688B204A643045FAA6480444CE09144E/C2CB6F3D18104B02B63CC3351D3E2252] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-01-14 16:29:51,808 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [688B204A643045FAA6480444CE09144E/C2CB6F3D18104B02B63CC3351D3E2252] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-01-14 16:29:51,810 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452788991665_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/688B204A643045FAA6480444CE09144E/C2CB6F3D18104B02B63CC3351D3E2252/.tez/application_1452788991665_0001/localmode-log-dir/dag_1452788991665_0001_1.dot
2016-01-14 16:29:51,810 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/688B204A643045FAA6480444CE09144E/C2CB6F3D18104B02B63CC3351D3E2252/.tez/application_1452788991665_0001/localmode-log-dir/dag_1452788991665_0001_1-tez-dag.pb.txt
2016-01-14 16:29:51,821 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788991665_0001_1
2016-01-14 16:29:51,821 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:51,821 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [688B204A643045FAA6480444CE09144E/C2CB6F3D18104B02B63CC3351D3E2252] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-01-14 16:29:51 Running Dag: dag_1452788991665_0001_1
2016-01-14 16:29:51,843 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452788991665_0001_1, submitTime=1452788991808
2016-01-14 16:29:51,843 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 16:29:51,845 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C503AF3DA2274DD49965CD09E3C6D47D
2016-01-14 16:29:51,846 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 16:29:51,846 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452788991665_0001_1, initTime=1452788991843
2016-01-14 16:29:51,847 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 16:29:51,847 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788991665_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 16:29:51,848 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:DAG_STARTED]: dagID=dag_1452788991665_0001_1, startTime=1452788991848
2016-01-14 16:29:51,848 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 16:29:51,848 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 16:29:51,849 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788991665_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 16:29:51,849 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: DC5F476C885A4A96AED930370EA900DA : {9AAA672F7F2642C5805FC0DC6F92EFE4={InputName=9AAA672F7F2642C5805FC0DC6F92EFE4}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 16:29:51,849 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 9AAA672F7F2642C5805FC0DC6F92EFE4, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 16:29:51,849 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452788991665_0001_1_00 [DC5F476C885A4A96AED930370EA900DA]
2016-01-14 16:29:51,850 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452788991665_0001_1_00 [DC5F476C885A4A96AED930370EA900DA]
2016-01-14 16:29:51,850 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452788991665_0001_1_00 [DC5F476C885A4A96AED930370EA900DA]
2016-01-14 16:29:51,850 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452788991665_0001_1_00 [DC5F476C885A4A96AED930370EA900DA]. Starting root input initializers: 1
2016-01-14 16:29:51,850 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...estCoGroup/complexcogroup, applicationId=application_1452788991665_0001, dagName=[688B204A643045FAA6480444CE09144E/C2CB6F3D18104B02B63CC3351D3E2252] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-01-14 16:29:51,856 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788991665_0001_1_00 [DC5F476C885A4A96AED930370EA900DA] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 16:29:51,856 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 9AAA672F7F2642C5805FC0DC6F92EFE4 on vertex vertex_1452788991665_0001_1_00 [DC5F476C885A4A96AED930370EA900DA]
2016-01-14 16:29:51,856 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FEFB6259CDD147DAB3BACC3D2414DA49 : {16A5BF02A0EF4A06A1CE6965046811E6={InputName=16A5BF02A0EF4A06A1CE6965046811E6}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 16:29:51,857 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 16A5BF02A0EF4A06A1CE6965046811E6, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 16:29:51,857 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452788991665_0001_1_01 [FEFB6259CDD147DAB3BACC3D2414DA49]
2016-01-14 16:29:51,857 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452788991665_0001_1_01 [FEFB6259CDD147DAB3BACC3D2414DA49]
2016-01-14 16:29:51,857 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452788991665_0001_1_01 [FEFB6259CDD147DAB3BACC3D2414DA49]
2016-01-14 16:29:51,857 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452788991665_0001_1_01 [FEFB6259CDD147DAB3BACC3D2414DA49]. Starting root input initializers: 1
2016-01-14 16:29:51,861 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 16:29:51,862 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 9AAA672F7F2642C5805FC0DC6F92EFE4 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 16:29:51,863 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788991665_0001_1_01 [FEFB6259CDD147DAB3BACC3D2414DA49] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 16:29:51,863 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452788991665_0001_1_02 [C503AF3DA2274DD49965CD09E3C6D47D]
2016-01-14 16:29:51,863 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 16:29:51,864 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 16:29:51,866 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-01-14 16:29:51,866 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 16:29:51,867 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452788991665_0001_1_02 [C503AF3DA2274DD49965CD09E3C6D47D]
2016-01-14 16:29:51,867 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452788991665_0001_1_02 [C503AF3DA2274DD49965CD09E3C6D47D]
2016-01-14 16:29:51,867 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452788991665_0001_1_02 [C503AF3DA2274DD49965CD09E3C6D47D]
2016-01-14 16:29:51,869 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=C9CB7C2DF987456E8CA4B695E38A8F2F, vertexId=vertex_1452788991665_0001_1_02 [C503AF3DA2274DD49965CD09E3C6D47D], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 16:29:51,870 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=C9CB7C2DF987456E8CA4B695E38A8F2F, vertexId=vertex_1452788991665_0001_1_02 [C503AF3DA2274DD49965CD09E3C6D47D]
2016-01-14 16:29:51,870 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 16A5BF02A0EF4A06A1CE6965046811E6 on vertex vertex_1452788991665_0001_1_01 [FEFB6259CDD147DAB3BACC3D2414DA49]
2016-01-14 16:29:51,871 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 9AAA672F7F2642C5805FC0DC6F92EFE4 on vertex vertex_1452788991665_0001_1_00 [DC5F476C885A4A96AED930370EA900DA]
2016-01-14 16:29:51,872 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=C9CB7C2DF987456E8CA4B695E38A8F2F, vertexName=C503AF3DA2274DD49965CD09E3C6D47D, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 16:29:51,872 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=C9CB7C2DF987456E8CA4B695E38A8F2F, vertexId=vertex_1452788991665_0001_1_02 [C503AF3DA2274DD49965CD09E3C6D47D]
2016-01-14 16:29:51,873 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 16:29:51,873 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 16A5BF02A0EF4A06A1CE6965046811E6 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 16:29:51,874 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:51,869 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] submitted tez dag to app master: application_1452788991665_0001, with dag id: dag_1452788991665_0001_1
2016-01-14 16:29:51,874 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C503AF3DA2274DD49965CD09E3C6D47D, vertexId=vertex_1452788991665_0001_1_02, initRequestedTime=1452788991863, initedTime=1452788991874, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 16:29:51,876 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788991665_0001_1_02 [C503AF3DA2274DD49965CD09E3C6D47D] transitioned from NEW to INITED due to event V_INIT
2016-01-14 16:29:51,877 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788991665_0001_1_00 [DC5F476C885A4A96AED930370EA900DA] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:51,877 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 16:29:51,879 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 16:29:51,880 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {9AAA672F7F2642C5805FC0DC6F92EFE4=forAllWorkUnits=true, update=[1]}
2016-01-14 16:29:51,880 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452788991665_0001_1_00 [DC5F476C885A4A96AED930370EA900DA] parallelism set to 1
2016-01-14 16:29:51,880 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788991665_0001_1_00 [DC5F476C885A4A96AED930370EA900DA] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:51,881 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-01-14 16:29:51,881 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788991665_0001_1_00, eventCount=1
2016-01-14 16:29:51,881 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 16A5BF02A0EF4A06A1CE6965046811E6 on vertex vertex_1452788991665_0001_1_01 [FEFB6259CDD147DAB3BACC3D2414DA49]
2016-01-14 16:29:51,881 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:51,881 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DC5F476C885A4A96AED930370EA900DA, vertexId=vertex_1452788991665_0001_1_00, initRequestedTime=1452788991849, initedTime=1452788991881, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 16:29:51,881 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788991665_0001_1_00 [DC5F476C885A4A96AED930370EA900DA] with distanceFromRoot: 0
2016-01-14 16:29:51,882 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788991665_0001_1_00 [DC5F476C885A4A96AED930370EA900DA] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 16:29:51,886 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788991665_0001_1_01 [FEFB6259CDD147DAB3BACC3D2414DA49] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:51,887 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {16A5BF02A0EF4A06A1CE6965046811E6=forAllWorkUnits=true, update=[1]}
2016-01-14 16:29:51,887 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452788991665_0001_1_01 [FEFB6259CDD147DAB3BACC3D2414DA49] parallelism set to 1
2016-01-14 16:29:51,888 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788991665_0001_1_01 [FEFB6259CDD147DAB3BACC3D2414DA49] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:51,888 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788991665_0001_1_01, eventCount=1
2016-01-14 16:29:51,889 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FEFB6259CDD147DAB3BACC3D2414DA49, vertexId=vertex_1452788991665_0001_1_01, initRequestedTime=1452788991856, initedTime=1452788991889, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 16:29:51,889 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:51,890 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788991665_0001_1_01 [FEFB6259CDD147DAB3BACC3D2414DA49] with distanceFromRoot: 0
2016-01-14 16:29:51,890 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788991665_0001_1_01 [FEFB6259CDD147DAB3BACC3D2414DA49] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 16:29:51,891 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in DC5F476C885A4A96AED930370EA900DA
2016-01-14 16:29:51,891 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788991665_0001_1_00 [DC5F476C885A4A96AED930370EA900DA]
2016-01-14 16:29:51,891 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:51,892 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788991665_0001_1_00, startRequestedTime=1452788991891, startedTime=1452788991891
2016-01-14 16:29:51,892 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788991665_0001_1_00 [DC5F476C885A4A96AED930370EA900DA] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:51,893 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FEFB6259CDD147DAB3BACC3D2414DA49
2016-01-14 16:29:51,893 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788991665_0001_1_01 [FEFB6259CDD147DAB3BACC3D2414DA49]
2016-01-14 16:29:51,894 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:51,894 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788991665_0001_1_01, startRequestedTime=1452788991893, startedTime=1452788991893
2016-01-14 16:29:51,894 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788991665_0001_1_01 [FEFB6259CDD147DAB3BACC3D2414DA49] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:51,895 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:51,896 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:TASK_STARTED]: vertexName=DC5F476C885A4A96AED930370EA900DA, taskId=task_1452788991665_0001_1_00_000000, scheduledTime=1452788991895, launchTime=1452788991895
2016-01-14 16:29:51,896 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788991665_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:51,896 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452788991665_0001_1_00 for vertex: vertex_1452788991665_0001_1_02 [C503AF3DA2274DD49965CD09E3C6D47D] numStartedSources: 1 numSources: 2
2016-01-14 16:29:51,897 INFO impl.VertexImpl (VertexImpl.java:transition(3160)) - Cannot start vertex: vertex_1452788991665_0001_1_02 [C503AF3DA2274DD49965CD09E3C6D47D] numStartedSources: 1 numSources: 2
2016-01-14 16:29:51,897 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:51,897 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:TASK_STARTED]: vertexName=FEFB6259CDD147DAB3BACC3D2414DA49, taskId=task_1452788991665_0001_1_01_000000, scheduledTime=1452788991897, launchTime=1452788991897
2016-01-14 16:29:51,898 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788991665_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:51,898 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452788991665_0001_1_01 for vertex: vertex_1452788991665_0001_1_02 [C503AF3DA2274DD49965CD09E3C6D47D] numStartedSources: 2 numSources: 2
2016-01-14 16:29:51,898 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788991665_0001_1_02 [C503AF3DA2274DD49965CD09E3C6D47D] with distanceFromRoot: 1
2016-01-14 16:29:51,898 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788991665_0001_1_00 [DC5F476C885A4A96AED930370EA900DA]
2016-01-14 16:29:51,899 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788991665_0001_1_01 [FEFB6259CDD147DAB3BACC3D2414DA49]
2016-01-14 16:29:51,899 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: FEFB6259CDD147DAB3BACC3D2414DA49 in vertex: C503AF3DA2274DD49965CD09E3C6D47D
2016-01-14 16:29:51,899 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: DC5F476C885A4A96AED930370EA900DA in vertex: C503AF3DA2274DD49965CD09E3C6D47D
2016-01-14 16:29:51,899 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: C503AF3DA2274DD49965CD09E3C6D47D with 2 source tasks and 1 pending tasks
2016-01-14 16:29:51,900 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:51,900 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788991665_0001_1_02, startRequestedTime=1452788991899, startedTime=1452788991899
2016-01-14 16:29:51,900 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788991665_0001_1_02 [C503AF3DA2274DD49965CD09E3C6D47D] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:51,901 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [688B204A643045FAA6480444CE09144E/C2CB6F3D18104B02B63CC3351D3E2252] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: DC5F476C885A4A96AED930370EA900DA, VertexParallelism: 1, TaskAttemptID:attempt_1452788991665_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9AAA672F7F2642C5805FC0DC6F92EFE4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C503AF3DA2274DD49965CD09E3C6D47D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 16:29:51,901 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788991665_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:51,901 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [688B204A643045FAA6480444CE09144E/C2CB6F3D18104B02B63CC3351D3E2252] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: FEFB6259CDD147DAB3BACC3D2414DA49, VertexParallelism: 1, TaskAttemptID:attempt_1452788991665_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=16A5BF02A0EF4A06A1CE6965046811E6, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C503AF3DA2274DD49965CD09E3C6D47D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 16:29:51,901 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788991665_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:51,902 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:51,902 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:51,902 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 16:29:51,903 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 16:29:51,903 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 16:29:51,903 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788991665_0001_00_000001
2016-01-14 16:29:51,904 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991665_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:51,904 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788991665_0001_00_000002
2016-01-14 16:29:51,904 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991665_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:51,906 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788991665_0001_00_000001
2016-01-14 16:29:51,906 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 16:29:51,908 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788991665_0001_00_000002
2016-01-14 16:29:51,908 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 16:29:51,909 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:51,909 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991665_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:51,909 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788991665_0001_00_000001, launchTime=1452788991909
2016-01-14 16:29:51,910 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:51,911 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788991665_0001_00_000001
2016-01-14 16:29:51,911 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991665_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:51,911 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788991665_0001_00_000002, launchTime=1452788991910
2016-01-14 16:29:51,911 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788991665_0001_1_00_000000_0] to container: [container_1452788991665_0001_00_000001]
2016-01-14 16:29:51,912 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991665_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:51,912 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788991665_0001_00_000002
2016-01-14 16:29:51,912 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788991665_0001_00_000001 given task: attempt_1452788991665_0001_1_00_000000_0
2016-01-14 16:29:51,915 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788991665_0001_1_00_000000_0] started. Is using containerId: [container_1452788991665_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 16:29:51,915 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DC5F476C885A4A96AED930370EA900DA, taskAttemptId=attempt_1452788991665_0001_1_00_000000_0, startTime=1452788991912, containerId=container_1452788991665_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788991665_0001_00_000001/teamcity, completedLogs=
2016-01-14 16:29:51,915 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788991665_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:51,915 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788991665_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:51,915 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788991665_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788991665_0001_1_00_000000_0
2016-01-14 16:29:51,915 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788991665_0001_1_01_000000_0] to container: [container_1452788991665_0001_00_000002]
2016-01-14 16:29:51 Starting to run new task attempt: attempt_1452788991665_0001_1_00_000000_0
2016-01-14 16:29:51,917 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788991665_0001_1_00_000000_0
2016-01-14 16:29:51,917 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:51,917 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:51,917 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:51,918 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:51,918 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:51,918 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:51,918 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [688B204A643045FAA6480444CE09144E/C2CB6F3D18104B02B63CC3351D3E2252] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: DC5F476C885A4A96AED930370EA900DA, VertexParallelism: 1, TaskAttemptID:attempt_1452788991665_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9AAA672F7F2642C5805FC0DC6F92EFE4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C503AF3DA2274DD49965CD09E3C6D47D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 16:29:51,918 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:51,915 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:51,919 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788991665_0001_1_00_000000_0
2016-01-14 16:29:51,919 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:51,919 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:51,920 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788991665_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 16:29:51,919 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991665_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:51,921 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788991665_0001_1_01_000000_0] started. Is using containerId: [container_1452788991665_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 16:29:51,921 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FEFB6259CDD147DAB3BACC3D2414DA49, taskAttemptId=attempt_1452788991665_0001_1_01_000000_0, startTime=1452788991921, containerId=container_1452788991665_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788991665_0001_00_000002/teamcity, completedLogs=
2016-01-14 16:29:51,921 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788991665_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:51,921 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788991665_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:51,922 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:51,922 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=9AAA672F7F2642C5805FC0DC6F92EFE4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 16:29:51,923 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:51,923 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 9AAA672F7F2642C5805FC0DC6F92EFE4
2016-01-14 16:29:51,923 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 9AAA672F7F2642C5805FC0DC6F92EFE4
2016-01-14 16:29:51,925 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:51,928 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:51,930 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788991665_0001_00_000002 given task: attempt_1452788991665_0001_1_01_000000_0
2016-01-14 16:29:51,930 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788991665_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788991665_0001_1_01_000000_0
2016-01-14 16:29:51 Starting to run new task attempt: attempt_1452788991665_0001_1_01_000000_0
2016-01-14 16:29:51,931 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788991665_0001_1_01_000000_0
2016-01-14 16:29:51,931 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:51,931 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:51,931 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:51,931 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:51,931 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:51,931 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:51,931 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [688B204A643045FAA6480444CE09144E/C2CB6F3D18104B02B63CC3351D3E2252] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: FEFB6259CDD147DAB3BACC3D2414DA49, VertexParallelism: 1, TaskAttemptID:attempt_1452788991665_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=16A5BF02A0EF4A06A1CE6965046811E6, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C503AF3DA2274DD49965CD09E3C6D47D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 16:29:51,932 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:51,933 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788991665_0001_1_01_000000_0
2016-01-14 16:29:51,933 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:51,933 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:51,935 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:51,949 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C503AF3DA2274DD49965CD09E3C6D47D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 16:29:51,949 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:51,950 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C503AF3DA2274DD49965CD09E3C6D47D
2016-01-14 16:29:51,950 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 16:29:51,950 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 9AAA672F7F2642C5805FC0DC6F92EFE4
2016-01-14 16:29:51,950 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 9AAA672F7F2642C5805FC0DC6F92EFE4
2016-01-14 16:29:51,951 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C503AF3DA2274DD49965CD09E3C6D47D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 16:29:51,951 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788991665_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 16:29:51,952 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:51,952 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C503AF3DA2274DD49965CD09E3C6D47D
2016-01-14 16:29:51,952 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=16A5BF02A0EF4A06A1CE6965046811E6, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 16:29:51,953 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:51,953 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 16A5BF02A0EF4A06A1CE6965046811E6
2016-01-14 16:29:51,955 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 16A5BF02A0EF4A06A1CE6965046811E6
2016-01-14 16:29:51,955 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 16:29:51,956 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C503AF3DA2274DD49965CD09E3C6D47D
2016-01-14 16:29:51,955 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DC5F476C885A4A96AED930370EA900DA, ordinal: 0
2016-01-14 16:29:51,955 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 16:29:51,960 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C503AF3DA2274DD49965CD09E3C6D47D
2016-01-14 16:29:51,959 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C503AF3DA2274DD49965CD09E3C6D47D
2016-01-14 16:29:51,964 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 16:29:51,965 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 16A5BF02A0EF4A06A1CE6965046811E6
2016-01-14 16:29:51,965 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 16A5BF02A0EF4A06A1CE6965046811E6
2016-01-14 16:29:51,962 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DC5F476C885A4A96AED930370EA900DA, mem on start (mb), free: 323, total: 356, max: 672
2016-01-14 16:29:51,961 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:51,961 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C503AF3DA2274DD49965CD09E3C6D47D
2016-01-14 16:29:51,969 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:51,970 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:51,970 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:51,970 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:51,970 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:51,971 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 16:29:51,971 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:51,971 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 16:29:51,971 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 16:29:51,972 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 16:29:51,972 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 9AAA672F7F2642C5805FC0DC6F92EFE4, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 16:29:51,972 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C503AF3DA2274DD49965CD09E3C6D47D, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 16:29:51,973 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:51,973 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 9AAA672F7F2642C5805FC0DC6F92EFE4 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:51,973 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 9AAA672F7F2642C5805FC0DC6F92EFE4
2016-01-14 16:29:51,973 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:51,973 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 9AAA672F7F2642C5805FC0DC6F92EFE4
2016-01-14 16:29:51,974 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 16:29:51,974 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:51,974 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:51,975 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788991665_0001_1_00_000000_0
2016-01-14 16:29:51,976 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FEFB6259CDD147DAB3BACC3D2414DA49, ordinal: 1
2016-01-14 16:29:51,976 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FEFB6259CDD147DAB3BACC3D2414DA49, mem on start (mb), free: 323, total: 356, max: 672
2016-01-14 16:29:51,976 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:51,976 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:51,976 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:51,977 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:51,977 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:51,977 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 16:29:51,977 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:51,977 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 16:29:51,978 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 16:29:51,978 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 16:29:51,978 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-01-14 16:29:51,979 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 16:29:51,979 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 16A5BF02A0EF4A06A1CE6965046811E6, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 16:29:51,980 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C503AF3DA2274DD49965CD09E3C6D47D, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 16:29:51,980 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:51,980 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 16A5BF02A0EF4A06A1CE6965046811E6
2016-01-14 16:29:51,980 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:51,980 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 16A5BF02A0EF4A06A1CE6965046811E6
2016-01-14 16:29:51,981 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 16A5BF02A0EF4A06A1CE6965046811E6 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:51,981 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 16:29:51,982 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:51,982 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:51,982 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt"] streamed: true, id: 9AAA672F7F2642C5805FC0DC6F92EFE4
2016-01-14 16:29:51,982 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788991665_0001_1_01_000000_0
2016-01-14 16:29:51,983 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 16:29:51,983 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@5a8fed6b
2016-01-14 16:29:51,984 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@10a44fbd
2016-01-14 16:29:51,985 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@5a8fed6b
2016-01-14 16:29:51,985 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 16:29:51,986 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 16:29:51,986 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]], id: FA5708B6FAD2495F8A87BBF203C2206C
2016-01-14 16:29:51,987 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] FA5708B6FAD2495F8A87BBF203C2206C
2016-01-14 16:29:51,987 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@687b5a18; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@7a58133e; comparator=cascading.tuple.hadoop.util.TupleComparator@2c578a32; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 16:29:51,986 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 16:29:51,986 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@10a44fbd
2016-01-14 16:29:51,988 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 16:29:51,988 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 16:29:51,988 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 16:29:51,988 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 16:29:51,989 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 16:29:51,988 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"] streamed: true, id: 16A5BF02A0EF4A06A1CE6965046811E6
2016-01-14 16:29:51,989 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]], id: FA5708B6FAD2495F8A87BBF203C2206C
2016-01-14 16:29:51,989 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] FA5708B6FAD2495F8A87BBF203C2206C
2016-01-14 16:29:51,990 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@6b6fa5e3; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@6ef4112f; comparator=cascading.tuple.hadoop.util.TupleComparator@59945d91; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 16:29:52,002 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 16:29:52,002 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 16:29:52,002 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 16:29:52,002 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 16:29:52,002 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 16:29:52,002 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 16:29:52,003 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 16:29:52,002 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 16:29:52,003 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 16:29:52,004 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 16:29:52,005 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-01-14 16:29:52,005 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:52,006 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DC5F476C885A4A96AED930370EA900DA, all 1 inputs ready in: 00:00:00.001
2016-01-14 16:29:52,017 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 16:29:52,017 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 16:29:52,018 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 16:29:52,018 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 16:29:52,018 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 16:29:52,019 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 16:29:52,019 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:52,019 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FEFB6259CDD147DAB3BACC3D2414DA49, all 1 inputs ready in: 00:00:00.000
2016-01-14 16:29:52,019 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788991665_0001_1_00_000000_0
2016-01-14 16:29:52,019 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DC5F476C885A4A96AED930370EA900DA, mem on close (mb), free: 120, total: 356, max: 672
2016-01-14 16:29:52,020 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 16:29:52,020 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 16:29:52,020 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-01-14 16:29:52,020 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-01-14 16:29:52,022 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788991665_0001_1_01_000000_0
2016-01-14 16:29:52,022 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FEFB6259CDD147DAB3BACC3D2414DA49, mem on close (mb), free: 120, total: 356, max: 672
2016-01-14 16:29:52,022 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 16:29:52,022 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 16:29:52,022 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-01-14 16:29:52,022 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-01-14 16:29:52,024 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 16:29:52,025 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 16:29:52 Completed running task attempt: attempt_1452788991665_0001_1_00_000000_0
2016-01-14 16:29:52,029 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788991665_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:52,030 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788991665_0001_1_00, eventCount=1
2016-01-14 16:29:52,030 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=92, FILE_BYTES_WRITTEN=254, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452788992006, Process_Duration=13, Process_End_Time=1452788992019, Read_Duration=13, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-01-14 16:29:52,031 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DC5F476C885A4A96AED930370EA900DA, taskAttemptId=attempt_1452788991665_0001_1_00_000000_0, startTime=1452788991912, finishTime=1452788992030, timeTaken=118, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=92, FILE_BYTES_WRITTEN=254, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788992006, Process_Duration=13, Process_End_Time=1452788992019, Read_Duration=13, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 16:29:52,031 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788991665_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:52,031 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788991665_0001_1_00_000000_0
2016-01-14 16:29:52,032 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:TASK_FINISHED]: vertexName=DC5F476C885A4A96AED930370EA900DA, taskId=task_1452788991665_0001_1_00_000000, startTime=1452788991912, finishTime=1452788992031, timeTaken=119, status=SUCCEEDED, successfulAttemptID=attempt_1452788991665_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=92, FILE_BYTES_WRITTEN=254, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788992006, Process_Duration=13, Process_End_Time=1452788992019, Read_Duration=13, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 16:29:52,032 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788991665_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:52,032 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788991665_0001_1_00 [DC5F476C885A4A96AED930370EA900DA] : 1
2016-01-14 16:29:52,032 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788991665_0001_1_00 [DC5F476C885A4A96AED930370EA900DA]
2016-01-14 16:29:52,033 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:52,033 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:52,033 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:52,034 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:52,035 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:52,035 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788991665_0001_00_000001
2016-01-14 16:29:52,035 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:VERTEX_FINISHED]: vertexName=DC5F476C885A4A96AED930370EA900DA, vertexId=vertex_1452788991665_0001_1_00, initRequestedTime=1452788991849, initedTime=1452788991881, startRequestedTime=1452788991891, startedTime=1452788991891, finishTime=1452788992032, timeTaken=141, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=92, FILE_BYTES_WRITTEN=254, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788992006, Process_Duration=13, Process_End_Time=1452788992019, Read_Duration=13, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1452788991912, firstTasksToStart=[ task_1452788991665_0001_1_00_000000 ], lastTaskFinishTime=1452788992030, lastTasksToFinish=[ task_1452788991665_0001_1_00_000000 ], minTaskDuration=118, maxTaskDuration=118, avgTaskDuration=118.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788991665_0001_1_00_000000 ], longestDurationTasks=[ task_1452788991665_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:52,036 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788991665_0001_1_00 [DC5F476C885A4A96AED930370EA900DA] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:52,036 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452788991665_0001_1_02 [C503AF3DA2274DD49965CD09E3C6D47D] attempt: attempt_1452788991665_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 16:29:52,036 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991665_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:52,036 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:52,036 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788991665_0001_1_00 [DC5F476C885A4A96AED930370EA900DA] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 16:29:52,038 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788991665_0001_00_000001, stoppedTime=1452788992037, exitStatus=0
2016-01-14 16:29:52,038 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991665_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:52,038 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:52,038 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452788991665_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 16:29:52,038 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991665_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 16:29:52 Completed running task attempt: attempt_1452788991665_0001_1_01_000000_0
2016-01-14 16:29:52,040 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788991665_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:52,041 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788991665_0001_1_01, eventCount=1
2016-01-14 16:29:52,041 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=8161, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452788992019, Process_Duration=2, Process_End_Time=1452788992021, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-01-14 16:29:52,041 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FEFB6259CDD147DAB3BACC3D2414DA49, taskAttemptId=attempt_1452788991665_0001_1_01_000000_0, startTime=1452788991921, finishTime=1452788992041, timeTaken=120, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=8161, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788992019, Process_Duration=2, Process_End_Time=1452788992021, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 16:29:52,042 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788991665_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:52,042 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:52,041 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:52,042 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788991665_0001_1_01_000000_0
2016-01-14 16:29:52,042 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:52,043 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:TASK_FINISHED]: vertexName=FEFB6259CDD147DAB3BACC3D2414DA49, taskId=task_1452788991665_0001_1_01_000000, startTime=1452788991921, finishTime=1452788992043, timeTaken=122, status=SUCCEEDED, successfulAttemptID=attempt_1452788991665_0001_1_01_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=8161, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788992019, Process_Duration=2, Process_End_Time=1452788992021, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 16:29:52,043 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:52,043 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:52,043 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788991665_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:52,044 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788991665_0001_1_01 [FEFB6259CDD147DAB3BACC3D2414DA49] : 1
2016-01-14 16:29:52,044 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788991665_0001_1_01 [FEFB6259CDD147DAB3BACC3D2414DA49]
2016-01-14 16:29:52,044 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788991665_0001_00_000002
2016-01-14 16:29:52,045 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:52,045 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:VERTEX_FINISHED]: vertexName=FEFB6259CDD147DAB3BACC3D2414DA49, vertexId=vertex_1452788991665_0001_1_01, initRequestedTime=1452788991856, initedTime=1452788991889, startRequestedTime=1452788991893, startedTime=1452788991893, finishTime=1452788992044, timeTaken=151, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=8161, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788992019, Process_Duration=2, Process_End_Time=1452788992021, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1452788991921, firstTasksToStart=[ task_1452788991665_0001_1_01_000000 ], lastTaskFinishTime=1452788992041, lastTasksToFinish=[ task_1452788991665_0001_1_01_000000 ], minTaskDuration=120, maxTaskDuration=120, avgTaskDuration=120.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788991665_0001_1_01_000000 ], longestDurationTasks=[ task_1452788991665_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:52,045 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788991665_0001_1_01 [FEFB6259CDD147DAB3BACC3D2414DA49] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:52,045 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991665_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:52,046 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788991665_0001_00_000002, stoppedTime=1452788992046, exitStatus=0
2016-01-14 16:29:52,046 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991665_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:52,046 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452788991665_0001_1_02 [C503AF3DA2274DD49965CD09E3C6D47D] attempt: attempt_1452788991665_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 16:29:52,047 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: C503AF3DA2274DD49965CD09E3C6D47D
2016-01-14 16:29:52,047 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788991665_0001_1_02 [C503AF3DA2274DD49965CD09E3C6D47D]
2016-01-14 16:29:52,048 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788991665_0001_1_01 [FEFB6259CDD147DAB3BACC3D2414DA49] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 16:29:52,048 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452788991665_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 16:29:52,046 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:52,049 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:TASK_STARTED]: vertexName=C503AF3DA2274DD49965CD09E3C6D47D, taskId=task_1452788991665_0001_1_02_000000, scheduledTime=1452788992048, launchTime=1452788992048
2016-01-14 16:29:52,049 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788991665_0001_1_02_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:52,049 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788991665_0001_1_02 [C503AF3DA2274DD49965CD09E3C6D47D]
2016-01-14 16:29:52,050 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991665_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 16:29:52,050 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [688B204A643045FAA6480444CE09144E/C2CB6F3D18104B02B63CC3351D3E2252] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: C503AF3DA2274DD49965CD09E3C6D47D, VertexParallelism: 1, TaskAttemptID:attempt_1452788991665_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FEFB6259CDD147DAB3BACC3D2414DA49, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=DC5F476C885A4A96AED930370EA900DA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C9CB7C2DF987456E8CA4B695E38A8F2F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:52,050 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788991665_0001_1_02_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:52,051 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:52,051 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788991665_0001_00_000003
2016-01-14 16:29:52,052 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991665_0001_00_000003 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:52,049 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452788991665_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 16:29:52,060 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788991665_0001_00_000001: 24 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 16:29:52,060 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452788991665_0001_00_000001, Exiting
2016-01-14 16:29:52,060 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788991665_0001_00_000001
2016-01-14 16:29:52,060 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788991665_0001_00_000001
2016-01-14 16:29:52,049 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:52,060 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788991665_0001_00_000003
2016-01-14 16:29:52,062 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788991665_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 16:29:52,059 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452788991665_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 16:29:52,062 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788991665_0001_00_000002: 18 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 16:29:52,062 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452788991665_0001_00_000002, Exiting
2016-01-14 16:29:52,062 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788991665_0001_00_000002
2016-01-14 16:29:52,062 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788991665_0001_00_000002
2016-01-14 16:29:52,062 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 16:29:52,063 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452788991665_0001_00_000001 completed successfully
2016-01-14 16:29:52,063 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991665_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 16:29:52,065 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788991665_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 16:29:52,066 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452788991665_0001_00_000002 completed successfully
2016-01-14 16:29:52,066 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991665_0001_00_000003 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:52,066 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991665_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 16:29:52,066 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:52,067 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788991665_0001_00_000003, launchTime=1452788992066
2016-01-14 16:29:52,067 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788991665_0001_00_000003
2016-01-14 16:29:52,068 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788991665_0001_1_02_000000_0] to container: [container_1452788991665_0001_00_000003]
2016-01-14 16:29:52,068 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991665_0001_00_000003 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:52,069 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788991665_0001_1_02_000000_0] started. Is using containerId: [container_1452788991665_0001_00_000003] on NM: [127.0.0.1:0]
2016-01-14 16:29:52,069 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C503AF3DA2274DD49965CD09E3C6D47D, taskAttemptId=attempt_1452788991665_0001_1_02_000000_0, startTime=1452788992069, containerId=container_1452788991665_0001_00_000003, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788991665_0001_00_000003/teamcity, completedLogs=
2016-01-14 16:29:52,069 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788991665_0001_1_02_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:52,070 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788991665_0001_1_02_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:52,073 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:52,075 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788991665_0001_00_000003 given task: attempt_1452788991665_0001_1_02_000000_0
2016-01-14 16:29:52,076 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788991665_0001_00_000003: 1 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788991665_0001_1_02_000000_0
2016-01-14 16:29:52 Starting to run new task attempt: attempt_1452788991665_0001_1_02_000000_0
2016-01-14 16:29:52,076 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788991665_0001_1_02_000000_0
2016-01-14 16:29:52,077 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:52,077 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:52,077 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:52,077 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:52,077 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:52,077 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:52,077 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [688B204A643045FAA6480444CE09144E/C2CB6F3D18104B02B63CC3351D3E2252] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: C503AF3DA2274DD49965CD09E3C6D47D, VertexParallelism: 1, TaskAttemptID:attempt_1452788991665_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FEFB6259CDD147DAB3BACC3D2414DA49, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=DC5F476C885A4A96AED930370EA900DA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C9CB7C2DF987456E8CA4B695E38A8F2F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:52,078 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:52,079 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788991665_0001_1_02_000000_0
2016-01-14 16:29:52,079 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:52,080 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:52,082 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788991665_0001_1_02_000000_0 sent events: (0-2)
2016-01-14 16:29:52,082 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=FEFB6259CDD147DAB3BACC3D2414DA49, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 16:29:52,083 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:52,086 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: FEFB6259CDD147DAB3BACC3D2414DA49
2016-01-14 16:29:52,087 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=DC5F476C885A4A96AED930370EA900DA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 16:29:52,088 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:52,088 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: DC5F476C885A4A96AED930370EA900DA
2016-01-14 16:29:52,089 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 16:29:52,089 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 16:29:52,090 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: FEFB6259CDD147DAB3BACC3D2414DA49
2016-01-14 16:29:52,090 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: FEFB6259CDD147DAB3BACC3D2414DA49
2016-01-14 16:29:52,089 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 16:29:52,091 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 16:29:52,091 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: DC5F476C885A4A96AED930370EA900DA
2016-01-14 16:29:52,091 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:52,092 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C9CB7C2DF987456E8CA4B695E38A8F2F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 16:29:52,093 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:52,093 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C9CB7C2DF987456E8CA4B695E38A8F2F
2016-01-14 16:29:52,094 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 16:29:52,094 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: C9CB7C2DF987456E8CA4B695E38A8F2F
2016-01-14 16:29:52,095 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:52,095 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: DC5F476C885A4A96AED930370EA900DA
2016-01-14 16:29:52,102 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C503AF3DA2274DD49965CD09E3C6D47D, ordinal: 2
2016-01-14 16:29:52,102 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C503AF3DA2274DD49965CD09E3C6D47D, mem on start (mb), free: 324, total: 356, max: 672
2016-01-14 16:29:52,102 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:52,102 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 3 initializers to finish
2016-01-14 16:29:52,103 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:52,103 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:52,111 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 16:29:52,111 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C9CB7C2DF987456E8CA4B695E38A8F2F
2016-01-14 16:29:52,111 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:52,111 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:52,112 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 16:29:52,112 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 16:29:52,112 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 16:29:52,112 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:52,112 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345
2016-01-14 16:29:52,112 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 1268357504, TotalRequestedScaled: 6.0881160192E8, TotalJVMHeap: 704643072, TotalAvailable: 461541212, TotalRequested/TotalJVMHeap:1.80
2016-01-14 16:29:52,112 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, FEFB6259CDD147DAB3BACC3D2414DA49, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-01-14 16:29:52,113 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, DC5F476C885A4A96AED930370EA900DA, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-01-14 16:29:52,113 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C9CB7C2DF987456E8CA4B695E38A8F2F, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 16:29:52,113 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:52,113 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: FEFB6259CDD147DAB3BACC3D2414DA49 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:52,113 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: FEFB6259CDD147DAB3BACC3D2414DA49
2016-01-14 16:29:52,113 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: DC5F476C885A4A96AED930370EA900DA
2016-01-14 16:29:52,114 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 16:29:52,114 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 16:29:52,113 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: DC5F476C885A4A96AED930370EA900DA being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:52,114 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: FEFB6259CDD147DAB3BACC3D2414DA49 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 16:29:52,115 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-01-14 16:29:52,115 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 16:29:52,115 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 16:29:52,118 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 2
2016-01-14 16:29:52,118 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 2 IOs to start
2016-01-14 16:29:52,118 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: FEFB6259CDD147DAB3BACC3D2414DA49
2016-01-14 16:29:52,118 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:52,121 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 16:29:52,121 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 16:29:52,122 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: DC5F476C885A4A96AED930370EA900DA with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 16:29:52,122 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-01-14 16:29:52,122 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 16:29:52,122 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 16:29:52,126 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: DC5F476C885A4A96AED930370EA900DA
2016-01-14 16:29:52,126 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:52,126 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788991665_0001_1_02_000000_0
2016-01-14 16:29:52,128 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452788991665_0001_1_00_000000_0_10056, runDuration: 78075, ]
2016-01-14 16:29:52,129 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452788991665_0001_1_01_000000_0_10057, runDuration: 75610, ]
2016-01-14 16:29:52,130 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452788991665_0001_1_01_000000_0_10057] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1452788992130, TimeTaken=1, Rate=0.09 MB/s
2016-01-14 16:29:52,130 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : FEFB6259CDD147DAB3BACC3D2414DA49
2016-01-14 16:29:52,130 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 16:29:52,130 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [FEFB6259CDD147DAB3BACC3D2414DA49] #9 in 1ms
2016-01-14 16:29:52,130 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]]
2016-01-14 16:29:52,130 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [FEFB6259CDD147DAB3BACC3D2414DA49] #9, status:false, isInterrupted:false
2016-01-14 16:29:52,131 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 16:29:52,131 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] streamed: true, id: FA5708B6FAD2495F8A87BBF203C2206C
2016-01-14 16:29:52,131 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-01-14 16:29:52,131 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup"], id: C9CB7C2DF987456E8CA4B695E38A8F2F
2016-01-14 16:29:52,131 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 16:29:52,132 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 16:29:52,132 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-01-14 16:29:52,132 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : FEFB6259CDD147DAB3BACC3D2414DA49
2016-01-14 16:29:52,132 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 16:29:52,133 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup"]
2016-01-14 16:29:52,133 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] FA5708B6FAD2495F8A87BBF203C2206C, for 2 inputs
2016-01-14 16:29:52,133 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] FA5708B6FAD2495F8A87BBF203C2206C, for 2 inputs
2016-01-14 16:29:52,136 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452788991665_0001_1_00_000000_0_10056] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1452788992134, TimeTaken=4, Rate=0.02 MB/s
2016-01-14 16:29:52,136 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : DC5F476C885A4A96AED930370EA900DA
2016-01-14 16:29:52,137 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 16:29:52,136 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(89)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2016-01-14 16:29:52,138 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(94)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2016-01-14 16:29:52,138 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [DC5F476C885A4A96AED930370EA900DA] #10 in 8ms
2016-01-14 16:29:52,138 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [DC5F476C885A4A96AED930370EA900DA] #10, status:false, isInterrupted:false
2016-01-14 16:29:52,140 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 16:29:52,140 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-01-14 16:29:52,140 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 16:29:52,140 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 16:29:52,141 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-01-14 16:29:52,141 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : DC5F476C885A4A96AED930370EA900DA
2016-01-14 16:29:52,141 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 16:29:52,141 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C503AF3DA2274DD49965CD09E3C6D47D, all 2 inputs ready in: 00:00:00.003
2016-01-14 16:29:52,142 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@18fa26e; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@7cf0e48e
2016-01-14 16:29:52,142 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@2b2d66b7; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@31b28e86
2016-01-14 16:29:52,145 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 16:29:52,145 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 16:29:52,145 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452788991665_0001_1_02_000000_0
2016-01-14 16:29:52,145 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452788991665_0001_1_02_000000_0 given a go for committing the task output.
2016-01-14 16:29:52,146 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788991665_0001_1_02_000000_0
2016-01-14 16:29:52,146 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C503AF3DA2274DD49965CD09E3C6D47D, mem on close (mb), free: 320, total: 356, max: 672
2016-01-14 16:29:52,146 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: FEFB6259CDD147DAB3BACC3D2414DA49
2016-01-14 16:29:52,147 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 16:29:52,147 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 16:29:52,147 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 16:29:52,147 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 16:29:52,147 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 16:29:52,148 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: DC5F476C885A4A96AED930370EA900DA
2016-01-14 16:29:52,149 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 16:29:52,149 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 16:29:52,150 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 16:29:52,150 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 16:29:52,150 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 16:29:52 Completed running task attempt: attempt_1452788991665_0001_1_02_000000_0
2016-01-14 16:29:52,152 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788991665_0001_1_02_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:52,154 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C503AF3DA2274DD49965CD09E3C6D47D, taskAttemptId=attempt_1452788991665_0001_1_02_000000_0, startTime=1452788992069, finishTime=1452788992153, timeTaken=84, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=17, MERGE_PHASE_TIME=20, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=16, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788992142, Process_Duration=4, Process_End_Time=1452788992146, Read_Duration=1, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-01-14 16:29:52,154 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788991665_0001_1_02_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:52,154 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788991665_0001_1_02_000000_0
2016-01-14 16:29:52,154 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:52,154 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:52,155 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:TASK_FINISHED]: vertexName=C503AF3DA2274DD49965CD09E3C6D47D, taskId=task_1452788991665_0001_1_02_000000, startTime=1452788992069, finishTime=1452788992154, timeTaken=85, status=SUCCEEDED, successfulAttemptID=attempt_1452788991665_0001_1_02_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=17, MERGE_PHASE_TIME=20, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=16, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788992142, Process_Duration=4, Process_End_Time=1452788992146, Read_Duration=1, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-01-14 16:29:52,155 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788991665_0001_1_02_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:52,155 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788991665_0001_1_02 [C503AF3DA2274DD49965CD09E3C6D47D] : 1
2016-01-14 16:29:52,155 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:52,155 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788991665_0001_1_02 [C503AF3DA2274DD49965CD09E3C6D47D]
2016-01-14 16:29:52,157 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:VERTEX_FINISHED]: vertexName=C503AF3DA2274DD49965CD09E3C6D47D, vertexId=vertex_1452788991665_0001_1_02, initRequestedTime=1452788991863, initedTime=1452788991874, startRequestedTime=1452788991899, startedTime=1452788991899, finishTime=1452788992155, timeTaken=256, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=17, MERGE_PHASE_TIME=20, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=16, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788992142, Process_Duration=4, Process_End_Time=1452788992146, Read_Duration=1, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1452788992069, firstTasksToStart=[ task_1452788991665_0001_1_02_000000 ], lastTaskFinishTime=1452788992153, lastTasksToFinish=[ task_1452788991665_0001_1_02_000000 ], minTaskDuration=84, maxTaskDuration=84, avgTaskDuration=84.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788991665_0001_1_02_000000 ], longestDurationTasks=[ task_1452788991665_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:52,157 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788991665_0001_1_02 [C503AF3DA2274DD49965CD09E3C6D47D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:52,157 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:52,157 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991665_0001_00_000003 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:52,158 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788991665_0001_00_000003, stoppedTime=1452788992157, exitStatus=0
2016-01-14 16:29:52,158 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991665_0001_00_000003 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:52,158 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788991665_0001_1_02 [C503AF3DA2274DD49965CD09E3C6D47D] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 16:29:52,158 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:52,158 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452788991665_0001_1
2016-01-14 16:29:52,159 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452788991665_0001_1
2016-01-14 16:29:52,159 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452788991665_0001_1_00 [DC5F476C885A4A96AED930370EA900DA]
2016-01-14 16:29:52,159 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452788991665_0001_1_01 [FEFB6259CDD147DAB3BACC3D2414DA49]
2016-01-14 16:29:52,159 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: C9CB7C2DF987456E8CA4B695E38A8F2F for vertex: vertex_1452788991665_0001_1_02 [C503AF3DA2274DD49965CD09E3C6D47D]
2016-01-14 16:29:52,161 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=17, MERGE_PHASE_TIME=20, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=16][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452788992142, Process_Duration=4, Process_End_Time=1452788992146, Read_Duration=1, Tuples_Read=10, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-01-14 16:29:52,163 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:52,163 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:52,163 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788991665_0001_00_000003
2016-01-14 16:29:52,163 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452788991665_0001_00_000003 is valid, but no longer registered, and will be killed
2016-01-14 16:29:52,164 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788991665_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 16:29:52,164 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452788991665_0001_00_000003, Exiting
2016-01-14 16:29:52,164 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788991665_0001_00_000003
2016-01-14 16:29:52,164 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788991665_0001_00_000003
2016-01-14 16:29:52,164 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788991665_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 16:29:52,164 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452788991665_0001_00_000003 completed successfully
2016-01-14 16:29:52,173 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452788991665_0001_1, queueSize=0
2016-01-14 16:29:52,173 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 16:29:52,173 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788991665_0001_1][Event:DAG_FINISHED]: dagId=dag_1452788991665_0001_1, startTime=1452788991848, finishTime=1452788992170, timeTaken=322, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=2, File System Counters, FILE_BYTES_READ=470, FILE_BYTES_WRITTEN=8467, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=20, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1121452032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=15, OUTPUT_BYTES=150, OUTPUT_BYTES_WITH_OVERHEAD=182, OUTPUT_BYTES_PHYSICAL=190, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=17, MERGE_PHASE_TIME=20, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=16, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4358366976167, Process_Duration=19, Process_End_Time=4358366976186, Read_Duration=16, Tuples_Read=20, Tuples_Written=15, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=5
2016-01-14 16:29:52,174 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452788991665_0001_1 finished with state: SUCCEEDED
2016-01-14 16:29:52,174 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788991665_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 16:29:52,174 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452788991665_0001_00_000003
2016-01-14 16:29:52,174 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788991665_0001_00_000003 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 16:29:52 Completed Dag: dag_1452788991665_0001_1
2016-01-14 16:29:52,175 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452788991665_0001_1, dagState=SUCCEEDED
2016-01-14 16:29:52,175 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788991665_0001_1_post
2016-01-14 16:29:52,176 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:52,176 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 16:29:52,183 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...estCoGroup/complexcogroup, applicationId=application_1452788991665_0001
2016-01-14 16:29:52,186 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 16:29:52,186 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 16:29:52,187 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 16:29:52,187 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 16:29:52,187 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 16:29:52,187 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 16:29:52,187 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 16:29:52,188 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 16:29:52,188 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 16:29:52,188 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 16:29:52,188 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 16:29:52,189 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 16:29:52,189 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 16:29:52,194 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/688B204A643045FAA6480444CE09144E/C2CB6F3D18104B02B63CC3351D3E2252/.tez/application_1452788991665_0001
2016-01-14 16:29:52,195 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] completed in: 00:00.547
2016-01-14 16:29:52,273 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup
2016-01-14 16:29:52,295 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 16:29:52,295 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 16:29:52,328 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-01-14 16:29:52,333 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-01-14 16:29:52,334 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 16:29:52,334 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 16:29:52,334 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 16:29:52,347 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-01-14 16:29:52,348 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace
2016-01-14 16:29:52,361 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 16:29:52,361 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 16:29:52,361 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace"]
2016-01-14 16:29:52,362 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 16:29:52,362 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 16:29:52,363 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 16:29:52,364 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...tform/testReplace/replace
2016-01-14 16:29:52,364 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 16:29:52,381 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 16:29:52,386 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:52,386 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 16:29:52,386 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 16:29:52,395 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DA8A7D96DB0644428F5DF2734480A93E/C5C418B3B9A147199C5FAAEBD0BF35FE/.tez/application_1452788992386_0001 doesn't exist and is created
2016-01-14 16:29:52,412 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 16:29:52,412 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 16:29:52,413 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DA8A7D96DB0644428F5DF2734480A93E/C5C418B3B9A147199C5FAAEBD0BF35FE/.tez/application_1452788992386_0001
2016-01-14 16:29:52,416 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452788992386_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:52,426 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 16:29:52,427 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 16:29:52,429 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 16:29:52,429 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 16:29:52,429 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DA8A7D96DB0644428F5DF2734480A93E/C5C418B3B9A147199C5FAAEBD0BF35FE/.tez/application_1452788992386_0001/localmode-log-dir
2016-01-14 16:29:52,430 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DA8A7D96DB0644428F5DF2734480A93E/C5C418B3B9A147199C5FAAEBD0BF35FE/.tez/application_1452788992386_0001/localmode-log-dir/history.txt.appattempt_1452788992386_0001_000000, maxErrors=10
2016-01-14 16:29:52,430 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 16:29:52,430 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452788992386_0001_000000, appSubmitTime=1452788992415, launchTime=1452788992415
2016-01-14 16:29:52,435 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 16:29:52,437 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 16:29:52,437 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:57733
2016-01-14 16:29:52,438 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 16:29:52,438 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 16:29:52,440 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 16:29:52,442 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452788992386_0001_000000, startTime=1452788992442
2016-01-14 16:29:52,442 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 16:29:52,442 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 16:29:52,492 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 16:29:52,492 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 16:29:52,512 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 16:29:52,512 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 16:29:52,513 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...tform/testReplace/replace, applicationId=application_1452788992386_0001, dagName=[DA8A7D96DB0644428F5DF2734480A93E/C5C418B3B9A147199C5FAAEBD0BF35FE] test/(1/1) ...tform/testReplace/replace
2016-01-14 16:29:52,530 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [DA8A7D96DB0644428F5DF2734480A93E/C5C418B3B9A147199C5FAAEBD0BF35FE] test/(1/1) ...tform/testReplace/replace
2016-01-14 16:29:52,531 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452788992386_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DA8A7D96DB0644428F5DF2734480A93E/C5C418B3B9A147199C5FAAEBD0BF35FE/.tez/application_1452788992386_0001/localmode-log-dir/dag_1452788992386_0001_1.dot
2016-01-14 16:29:52,532 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DA8A7D96DB0644428F5DF2734480A93E/C5C418B3B9A147199C5FAAEBD0BF35FE/.tez/application_1452788992386_0001/localmode-log-dir/dag_1452788992386_0001_1-tez-dag.pb.txt
2016-01-14 16:29:52,537 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788992386_0001_1
2016-01-14 16:29:52,537 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:52,538 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [DA8A7D96DB0644428F5DF2734480A93E/C5C418B3B9A147199C5FAAEBD0BF35FE] test/(1/1) ...tform/testReplace/replace
2016-01-14 16:29:52 Running Dag: dag_1452788992386_0001_1
2016-01-14 16:29:52,552 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 16:29:52,552 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992386_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452788992386_0001_1, submitTime=1452788992530
2016-01-14 16:29:52,553 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D95FF6AA595C446DB3664807DC1D827F
2016-01-14 16:29:52,553 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 16:29:52,554 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 16:29:52,554 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992386_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452788992386_0001_1, initTime=1452788992552
2016-01-14 16:29:52,555 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788992386_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 16:29:52,555 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992386_0001_1][Event:DAG_STARTED]: dagID=dag_1452788992386_0001_1, startTime=1452788992555
2016-01-14 16:29:52,555 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 16:29:52,555 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 16:29:52,555 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788992386_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 16:29:52,555 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D95FF6AA595C446DB3664807DC1D827F : {4D6C03A36336469BA2143FC5CED1C376={InputName=4D6C03A36336469BA2143FC5CED1C376}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 16:29:52,556 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 4D6C03A36336469BA2143FC5CED1C376, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 16:29:52,556 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452788992386_0001_1_00 [D95FF6AA595C446DB3664807DC1D827F]
2016-01-14 16:29:52,556 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452788992386_0001_1_00 [D95FF6AA595C446DB3664807DC1D827F]
2016-01-14 16:29:52,556 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452788992386_0001_1_00 [D95FF6AA595C446DB3664807DC1D827F]
2016-01-14 16:29:52,556 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452788992386_0001_1_00 [D95FF6AA595C446DB3664807DC1D827F]. Starting root input initializers: 1
2016-01-14 16:29:52,557 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788992386_0001_1_00 [D95FF6AA595C446DB3664807DC1D827F] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 16:29:52,558 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 4D6C03A36336469BA2143FC5CED1C376 on vertex vertex_1452788992386_0001_1_00 [D95FF6AA595C446DB3664807DC1D827F]
2016-01-14 16:29:52,560 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...tform/testReplace/replace, applicationId=application_1452788992386_0001, dagName=[DA8A7D96DB0644428F5DF2734480A93E/C5C418B3B9A147199C5FAAEBD0BF35FE] test/(1/1) ...tform/testReplace/replace
2016-01-14 16:29:52,561 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 16:29:52,561 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 4D6C03A36336469BA2143FC5CED1C376 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 16:29:52,562 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 16:29:52,562 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 16:29:52,564 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-01-14 16:29:52,564 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 4D6C03A36336469BA2143FC5CED1C376 on vertex vertex_1452788992386_0001_1_00 [D95FF6AA595C446DB3664807DC1D827F]
2016-01-14 16:29:52,566 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788992386_0001_1_00 [D95FF6AA595C446DB3664807DC1D827F] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:52,566 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {4D6C03A36336469BA2143FC5CED1C376=forAllWorkUnits=true, update=[1]}
2016-01-14 16:29:52,566 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452788992386_0001_1_00 [D95FF6AA595C446DB3664807DC1D827F] parallelism set to 1
2016-01-14 16:29:52,566 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788992386_0001_1_00 [D95FF6AA595C446DB3664807DC1D827F] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:52,567 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992386_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788992386_0001_1_00, eventCount=1
2016-01-14 16:29:52,567 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452788992386_0001_1_00 [D95FF6AA595C446DB3664807DC1D827F]
2016-01-14 16:29:52,567 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=E36EC3DE6980428197A1596F69C164A1, vertexId=vertex_1452788992386_0001_1_00 [D95FF6AA595C446DB3664807DC1D827F], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 16:29:52,567 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=E36EC3DE6980428197A1596F69C164A1, vertexId=vertex_1452788992386_0001_1_00 [D95FF6AA595C446DB3664807DC1D827F]
2016-01-14 16:29:52,569 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=E36EC3DE6980428197A1596F69C164A1, vertexName=D95FF6AA595C446DB3664807DC1D827F, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 16:29:52,569 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=E36EC3DE6980428197A1596F69C164A1, vertexId=vertex_1452788992386_0001_1_00 [D95FF6AA595C446DB3664807DC1D827F]
2016-01-14 16:29:52,574 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992386_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D95FF6AA595C446DB3664807DC1D827F, vertexId=vertex_1452788992386_0001_1_00, initRequestedTime=1452788992555, initedTime=1452788992574, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 16:29:52,574 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788992386_0001_1_00 [D95FF6AA595C446DB3664807DC1D827F] with distanceFromRoot: 0
2016-01-14 16:29:52,574 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:52,575 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788992386_0001_1_00 [D95FF6AA595C446DB3664807DC1D827F] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 16:29:52,575 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D95FF6AA595C446DB3664807DC1D827F
2016-01-14 16:29:52,575 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788992386_0001_1_00 [D95FF6AA595C446DB3664807DC1D827F]
2016-01-14 16:29:52,577 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:52,577 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992386_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788992386_0001_1_00, startRequestedTime=1452788992575, startedTime=1452788992575
2016-01-14 16:29:52,577 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788992386_0001_1_00 [D95FF6AA595C446DB3664807DC1D827F] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:52,578 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:52,578 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992386_0001_1][Event:TASK_STARTED]: vertexName=D95FF6AA595C446DB3664807DC1D827F, taskId=task_1452788992386_0001_1_00_000000, scheduledTime=1452788992578, launchTime=1452788992578
2016-01-14 16:29:52,578 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788992386_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:52,579 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788992386_0001_1_00 [D95FF6AA595C446DB3664807DC1D827F]
2016-01-14 16:29:52,579 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [DA8A7D96DB0644428F5DF2734480A93E/C5C418B3B9A147199C5FAAEBD0BF35FE] test/(1/1) ...tform/testReplace/replace, VertexName: D95FF6AA595C446DB3664807DC1D827F, VertexParallelism: 1, TaskAttemptID:attempt_1452788992386_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4D6C03A36336469BA2143FC5CED1C376, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E36EC3DE6980428197A1596F69C164A1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:52,579 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788992386_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:52,580 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:52,581 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 16:29:52,581 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452788992386_0001, with dag id: dag_1452788992386_0001_1
2016-01-14 16:29:52,581 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 16:29:52,582 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 16:29:52,582 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788992386_0001_00_000001
2016-01-14 16:29:52,582 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788992386_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:52,585 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788992386_0001_00_000001
2016-01-14 16:29:52,585 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 16:29:52,586 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788992386_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:52,587 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:52,587 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992386_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788992386_0001_00_000001, launchTime=1452788992586
2016-01-14 16:29:52,587 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788992386_0001_00_000001
2016-01-14 16:29:52,588 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788992386_0001_1_00_000000_0] to container: [container_1452788992386_0001_00_000001]
2016-01-14 16:29:52,588 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788992386_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:52,589 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788992386_0001_1_00_000000_0] started. Is using containerId: [container_1452788992386_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 16:29:52,589 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992386_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D95FF6AA595C446DB3664807DC1D827F, taskAttemptId=attempt_1452788992386_0001_1_00_000000_0, startTime=1452788992589, containerId=container_1452788992386_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788992386_0001_00_000001/teamcity, completedLogs=
2016-01-14 16:29:52,589 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788992386_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:52,589 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:52,589 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788992386_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:52,590 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788992386_0001_00_000001 given task: attempt_1452788992386_0001_1_00_000000_0
2016-01-14 16:29:52,590 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788992386_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788992386_0001_1_00_000000_0
2016-01-14 16:29:52 Starting to run new task attempt: attempt_1452788992386_0001_1_00_000000_0
2016-01-14 16:29:52,591 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788992386_0001_1_00_000000_0
2016-01-14 16:29:52,591 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:52,591 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:52,591 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:52,591 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:52,592 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:52,592 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:52,592 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [DA8A7D96DB0644428F5DF2734480A93E/C5C418B3B9A147199C5FAAEBD0BF35FE] test/(1/1) ...tform/testReplace/replace, VertexName: D95FF6AA595C446DB3664807DC1D827F, VertexParallelism: 1, TaskAttemptID:attempt_1452788992386_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4D6C03A36336469BA2143FC5CED1C376, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E36EC3DE6980428197A1596F69C164A1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:52,592 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:52,593 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788992386_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 16:29:52,595 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788992386_0001_1_00_000000_0
2016-01-14 16:29:52,595 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:52,596 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:52,596 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:52,596 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=4D6C03A36336469BA2143FC5CED1C376, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 16:29:52,597 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:52,597 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 4D6C03A36336469BA2143FC5CED1C376
2016-01-14 16:29:52,597 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 4D6C03A36336469BA2143FC5CED1C376
2016-01-14 16:29:52,599 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 16:29:52,599 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:52,600 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 4D6C03A36336469BA2143FC5CED1C376
2016-01-14 16:29:52,600 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 4D6C03A36336469BA2143FC5CED1C376
2016-01-14 16:29:52,601 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E36EC3DE6980428197A1596F69C164A1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 16:29:52,602 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:52,602 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E36EC3DE6980428197A1596F69C164A1
2016-01-14 16:29:52,602 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 16:29:52,602 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: E36EC3DE6980428197A1596F69C164A1
2016-01-14 16:29:52,607 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D95FF6AA595C446DB3664807DC1D827F, ordinal: 0
2016-01-14 16:29:52,607 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D95FF6AA595C446DB3664807DC1D827F, mem on start (mb), free: 247, total: 356, max: 672
2016-01-14 16:29:52,607 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:52,607 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:52,607 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:52,612 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 16:29:52,612 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E36EC3DE6980428197A1596F69C164A1
2016-01-14 16:29:52,613 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:52,613 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:52,614 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 16:29:52,614 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:52,614 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 16:29:52,614 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:52,614 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 16:29:52,615 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-01-14 16:29:52,615 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 4D6C03A36336469BA2143FC5CED1C376, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 16:29:52,615 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E36EC3DE6980428197A1596F69C164A1, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 16:29:52,615 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:52,615 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 4D6C03A36336469BA2143FC5CED1C376
2016-01-14 16:29:52,615 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:52,616 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 4D6C03A36336469BA2143FC5CED1C376
2016-01-14 16:29:52,616 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 4D6C03A36336469BA2143FC5CED1C376 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:52,616 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 16:29:52,616 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:52,617 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:52,617 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788992386_0001_1_00_000000_0
2016-01-14 16:29:52,617 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 16:29:52,618 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@4dcc66d1
2016-01-14 16:29:52,619 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@4dcc66d1
2016-01-14 16:29:52,619 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 16:29:52,619 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 16:29:52,620 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 16:29:52,621 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 4D6C03A36336469BA2143FC5CED1C376
2016-01-14 16:29:52,621 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace"], id: E36EC3DE6980428197A1596F69C164A1
2016-01-14 16:29:52,622 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace"]
2016-01-14 16:29:52,622 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 16:29:52,622 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:52,622 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D95FF6AA595C446DB3664807DC1D827F, all 1 inputs ready in: 00:00:00.000
2016-01-14 16:29:52,624 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 16:29:52,625 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 16:29:52,625 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452788992386_0001_1_00_000000_0
2016-01-14 16:29:52,626 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452788992386_0001_1_00_000000_0 given a go for committing the task output.
2016-01-14 16:29:52,626 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788992386_0001_1_00_000000_0
2016-01-14 16:29:52,627 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D95FF6AA595C446DB3664807DC1D827F, mem on close (mb), free: 246, total: 356, max: 672
2016-01-14 16:29:52 Completed running task attempt: attempt_1452788992386_0001_1_00_000000_0
2016-01-14 16:29:52,628 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788992386_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:52,629 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992386_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D95FF6AA595C446DB3664807DC1D827F, taskAttemptId=attempt_1452788992386_0001_1_00_000000_0, startTime=1452788992589, finishTime=1452788992629, timeTaken=40, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452788992622, Process_Duration=4, Process_End_Time=1452788992626, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-01-14 16:29:52,629 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788992386_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:52,630 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:52,630 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788992386_0001_1_00_000000_0
2016-01-14 16:29:52,630 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:52,631 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992386_0001_1][Event:TASK_FINISHED]: vertexName=D95FF6AA595C446DB3664807DC1D827F, taskId=task_1452788992386_0001_1_00_000000, startTime=1452788992589, finishTime=1452788992631, timeTaken=42, status=SUCCEEDED, successfulAttemptID=attempt_1452788992386_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452788992622, Process_Duration=4, Process_End_Time=1452788992626, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-01-14 16:29:52,631 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788992386_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:52,631 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:52,632 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788992386_0001_1_00 [D95FF6AA595C446DB3664807DC1D827F] : 1
2016-01-14 16:29:52,632 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788992386_0001_1_00 [D95FF6AA595C446DB3664807DC1D827F]
2016-01-14 16:29:52,633 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:52,633 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992386_0001_1][Event:VERTEX_FINISHED]: vertexName=D95FF6AA595C446DB3664807DC1D827F, vertexId=vertex_1452788992386_0001_1_00, initRequestedTime=1452788992555, initedTime=1452788992574, startRequestedTime=1452788992575, startedTime=1452788992575, finishTime=1452788992632, timeTaken=57, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452788992622, Process_Duration=4, Process_End_Time=1452788992626, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10, vertexStats=firstTaskStartTime=1452788992589, firstTasksToStart=[ task_1452788992386_0001_1_00_000000 ], lastTaskFinishTime=1452788992629, lastTasksToFinish=[ task_1452788992386_0001_1_00_000000 ], minTaskDuration=40, maxTaskDuration=40, avgTaskDuration=40.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788992386_0001_1_00_000000 ], longestDurationTasks=[ task_1452788992386_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:52,633 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788992386_0001_1_00 [D95FF6AA595C446DB3664807DC1D827F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:52,633 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788992386_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:52,633 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992386_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788992386_0001_00_000001, stoppedTime=1452788992633, exitStatus=0
2016-01-14 16:29:52,634 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:52,634 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788992386_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:52,634 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788992386_0001_1_00 [D95FF6AA595C446DB3664807DC1D827F] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-01-14 16:29:52,634 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452788992386_0001_1
2016-01-14 16:29:52,634 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10][cascading.flow.SliceCounters Process_Begin_Time=1452788992622, Process_Duration=4, Process_End_Time=1452788992626, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10, Tuples_Written=10]]
2016-01-14 16:29:52,634 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992386_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452788992386_0001_1
2016-01-14 16:29:52,634 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:52,635 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:52,635 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: E36EC3DE6980428197A1596F69C164A1 for vertex: vertex_1452788992386_0001_1_00 [D95FF6AA595C446DB3664807DC1D827F]
2016-01-14 16:29:52,635 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788992386_0001_00_000001
2016-01-14 16:29:52,635 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452788992386_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 16:29:52,638 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788992386_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 16:29:52,638 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452788992386_0001_00_000001, Exiting
2016-01-14 16:29:52,638 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788992386_0001_00_000001
2016-01-14 16:29:52,638 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788992386_0001_00_000001
2016-01-14 16:29:52,640 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788992386_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 16:29:52,640 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452788992386_0001_00_000001 completed successfully
2016-01-14 16:29:52,644 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452788992386_0001_1, queueSize=0
2016-01-14 16:29:52,644 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 16:29:52,644 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992386_0001_1][Event:DAG_FINISHED]: dagId=dag_1452788992386_0001_1, startTime=1452788992555, finishTime=1452788992643, timeTaken=88, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452788992622, Process_Duration=4, Process_End_Time=1452788992626, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-01-14 16:29:52,645 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452788992386_0001_1 finished with state: SUCCEEDED
2016-01-14 16:29:52,645 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788992386_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 16:29:52,645 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452788992386_0001_00_000001
2016-01-14 16:29:52,645 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788992386_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 16:29:52 Completed Dag: dag_1452788992386_0001_1
2016-01-14 16:29:52,645 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452788992386_0001_1, dagState=SUCCEEDED
2016-01-14 16:29:52,645 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788992386_0001_1_post
2016-01-14 16:29:52,646 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:52,646 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 16:29:52,651 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...tform/testReplace/replace, applicationId=application_1452788992386_0001
2016-01-14 16:29:52,653 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 16:29:52,653 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 16:29:52,653 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 16:29:52,653 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 16:29:52,653 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 16:29:52,654 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 16:29:52,654 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 16:29:52,655 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 16:29:52,655 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 16:29:52,656 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 16:29:52,656 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 16:29:52,656 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 16:29:52,657 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 16:29:52,661 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DA8A7D96DB0644428F5DF2734480A93E/C5C418B3B9A147199C5FAAEBD0BF35FE/.tez/application_1452788992386_0001
2016-01-14 16:29:52,662 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:00.300
2016-01-14 16:29:52,696 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testReplace/replace
2016-01-14 16:29:52,718 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 16:29:52,718 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 16:29:52,758 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-01-14 16:29:52,760 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-01-14 16:29:52,761 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 16:29:52,761 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 16:29:52,761 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 16:29:52,775 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-01-14 16:29:52,781 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2
2016-01-14 16:29:52,824 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] starting
2016-01-14 16:29:52,825 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 16:29:52,825 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2"]
2016-01-14 16:29:52,825 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] parallel execution of steps is enabled: false
2016-01-14 16:29:52,825 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executing total steps: 1
2016-01-14 16:29:52,826 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] allocating management threads: 1
2016-01-14 16:29:52,826 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] starting step: (1/1) ...latform/testCount2/count2
2016-01-14 16:29:52,827 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 16:29:52,838 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tez session mode enabled: true
2016-01-14 16:29:52,843 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:52,843 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 16:29:52,843 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 16:29:52,852 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E8565EEA09A7493083CBF5CEA2006ACA/EE1EB61F3CF04BE7B2685A4D01379396/.tez/application_1452788992843_0001 doesn't exist and is created
2016-01-14 16:29:52,869 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 16:29:52,869 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 16:29:52,869 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E8565EEA09A7493083CBF5CEA2006ACA/EE1EB61F3CF04BE7B2685A4D01379396/.tez/application_1452788992843_0001
2016-01-14 16:29:52,872 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452788992843_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:52,882 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 16:29:52,882 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 16:29:52,884 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 16:29:52,884 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 16:29:52,884 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E8565EEA09A7493083CBF5CEA2006ACA/EE1EB61F3CF04BE7B2685A4D01379396/.tez/application_1452788992843_0001/localmode-log-dir
2016-01-14 16:29:52,885 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E8565EEA09A7493083CBF5CEA2006ACA/EE1EB61F3CF04BE7B2685A4D01379396/.tez/application_1452788992843_0001/localmode-log-dir/history.txt.appattempt_1452788992843_0001_000000, maxErrors=10
2016-01-14 16:29:52,885 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 16:29:52,885 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452788992843_0001_000000, appSubmitTime=1452788992872, launchTime=1452788992872
2016-01-14 16:29:52,889 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 16:29:52,895 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 16:29:52,896 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 16:29:52,897 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 16:29:52,900 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:60709
2016-01-14 16:29:52,902 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 16:29:52,904 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452788992843_0001_000000, startTime=1452788992904
2016-01-14 16:29:52,904 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 16:29:52,904 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 16:29:52,969 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 16:29:52,969 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 16:29:52,969 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testCount2/count2, applicationId=application_1452788992843_0001, dagName=[E8565EEA09A7493083CBF5CEA2006ACA/EE1EB61F3CF04BE7B2685A4D01379396] count/(1/1) ...latform/testCount2/count2
2016-01-14 16:29:52,986 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [E8565EEA09A7493083CBF5CEA2006ACA/EE1EB61F3CF04BE7B2685A4D01379396] count/(1/1) ...latform/testCount2/count2
2016-01-14 16:29:52,988 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452788992843_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E8565EEA09A7493083CBF5CEA2006ACA/EE1EB61F3CF04BE7B2685A4D01379396/.tez/application_1452788992843_0001/localmode-log-dir/dag_1452788992843_0001_1.dot
2016-01-14 16:29:52,988 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E8565EEA09A7493083CBF5CEA2006ACA/EE1EB61F3CF04BE7B2685A4D01379396/.tez/application_1452788992843_0001/localmode-log-dir/dag_1452788992843_0001_1-tez-dag.pb.txt
2016-01-14 16:29:52,996 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788992843_0001_1
2016-01-14 16:29:52,996 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:52,996 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [E8565EEA09A7493083CBF5CEA2006ACA/EE1EB61F3CF04BE7B2685A4D01379396] count/(1/1) ...latform/testCount2/count2
2016-01-14 16:29:52 Running Dag: dag_1452788992843_0001_1
2016-01-14 16:29:53,012 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992843_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452788992843_0001_1, submitTime=1452788992986
2016-01-14 16:29:53,012 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 16:29:53,013 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C8139514E0A44FAAA5C3DE0F95F5617B
2016-01-14 16:29:53,013 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 16:29:53,013 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 16:29:53,013 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992843_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452788992843_0001_1, initTime=1452788993012
2016-01-14 16:29:53,014 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788992843_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 16:29:53,014 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992843_0001_1][Event:DAG_STARTED]: dagID=dag_1452788992843_0001_1, startTime=1452788993014
2016-01-14 16:29:53,014 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 16:29:53,014 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788992843_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 16:29:53,015 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FF732053E15E437C9E4FDB3854D9720A : {89E2D24A5FB04F37A8345BDBAA483F71={InputName=89E2D24A5FB04F37A8345BDBAA483F71}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 16:29:53,015 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 89E2D24A5FB04F37A8345BDBAA483F71, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 16:29:53,015 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452788992843_0001_1_00 [FF732053E15E437C9E4FDB3854D9720A]
2016-01-14 16:29:53,015 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452788992843_0001_1_00 [FF732053E15E437C9E4FDB3854D9720A]
2016-01-14 16:29:53,015 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452788992843_0001_1_00 [FF732053E15E437C9E4FDB3854D9720A]
2016-01-14 16:29:53,016 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testCount2/count2, applicationId=application_1452788992843_0001, dagName=[E8565EEA09A7493083CBF5CEA2006ACA/EE1EB61F3CF04BE7B2685A4D01379396] count/(1/1) ...latform/testCount2/count2
2016-01-14 16:29:53,016 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452788992843_0001_1_00 [FF732053E15E437C9E4FDB3854D9720A]. Starting root input initializers: 1
2016-01-14 16:29:53,016 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788992843_0001_1_00 [FF732053E15E437C9E4FDB3854D9720A] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 16:29:53,016 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 89E2D24A5FB04F37A8345BDBAA483F71 on vertex vertex_1452788992843_0001_1_00 [FF732053E15E437C9E4FDB3854D9720A]
2016-01-14 16:29:53,016 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 16:29:53,016 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452788992843_0001_1_01 [C8139514E0A44FAAA5C3DE0F95F5617B]
2016-01-14 16:29:53,018 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 16:29:53,018 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 89E2D24A5FB04F37A8345BDBAA483F71 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 16:29:53,019 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 16:29:53,019 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 16:29:53,020 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 16:29:53,020 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452788992843_0001_1_01 [C8139514E0A44FAAA5C3DE0F95F5617B]
2016-01-14 16:29:53,020 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452788992843_0001_1_01 [C8139514E0A44FAAA5C3DE0F95F5617B]
2016-01-14 16:29:53,021 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452788992843_0001_1_01 [C8139514E0A44FAAA5C3DE0F95F5617B]
2016-01-14 16:29:53,021 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=5EEB1893C29648919A77CA139100726A, vertexId=vertex_1452788992843_0001_1_01 [C8139514E0A44FAAA5C3DE0F95F5617B], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 16:29:53,021 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=5EEB1893C29648919A77CA139100726A, vertexId=vertex_1452788992843_0001_1_01 [C8139514E0A44FAAA5C3DE0F95F5617B]
2016-01-14 16:29:53,021 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-01-14 16:29:53,022 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 89E2D24A5FB04F37A8345BDBAA483F71 on vertex vertex_1452788992843_0001_1_00 [FF732053E15E437C9E4FDB3854D9720A]
2016-01-14 16:29:53,023 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=5EEB1893C29648919A77CA139100726A, vertexName=C8139514E0A44FAAA5C3DE0F95F5617B, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 16:29:53,024 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=5EEB1893C29648919A77CA139100726A, vertexId=vertex_1452788992843_0001_1_01 [C8139514E0A44FAAA5C3DE0F95F5617B]
2016-01-14 16:29:53,025 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:53,026 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992843_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C8139514E0A44FAAA5C3DE0F95F5617B, vertexId=vertex_1452788992843_0001_1_01, initRequestedTime=1452788993016, initedTime=1452788993025, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 16:29:53,026 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788992843_0001_1_01 [C8139514E0A44FAAA5C3DE0F95F5617B] transitioned from NEW to INITED due to event V_INIT
2016-01-14 16:29:53,026 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788992843_0001_1_00 [FF732053E15E437C9E4FDB3854D9720A] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:53,026 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {89E2D24A5FB04F37A8345BDBAA483F71=forAllWorkUnits=true, update=[1]}
2016-01-14 16:29:53,027 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452788992843_0001_1_00 [FF732053E15E437C9E4FDB3854D9720A] parallelism set to 1
2016-01-14 16:29:53,027 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788992843_0001_1_00 [FF732053E15E437C9E4FDB3854D9720A] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:53,027 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992843_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788992843_0001_1_00, eventCount=1
2016-01-14 16:29:53,028 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:53,028 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992843_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FF732053E15E437C9E4FDB3854D9720A, vertexId=vertex_1452788992843_0001_1_00, initRequestedTime=1452788993015, initedTime=1452788993028, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 16:29:53,028 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788992843_0001_1_00 [FF732053E15E437C9E4FDB3854D9720A] with distanceFromRoot: 0
2016-01-14 16:29:53,029 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788992843_0001_1_00 [FF732053E15E437C9E4FDB3854D9720A] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 16:29:53,029 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FF732053E15E437C9E4FDB3854D9720A
2016-01-14 16:29:53,029 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788992843_0001_1_00 [FF732053E15E437C9E4FDB3854D9720A]
2016-01-14 16:29:53,030 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:53,030 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992843_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788992843_0001_1_00, startRequestedTime=1452788993029, startedTime=1452788993029
2016-01-14 16:29:53,030 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788992843_0001_1_00 [FF732053E15E437C9E4FDB3854D9720A] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:53,031 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:53,031 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992843_0001_1][Event:TASK_STARTED]: vertexName=FF732053E15E437C9E4FDB3854D9720A, taskId=task_1452788992843_0001_1_00_000000, scheduledTime=1452788993030, launchTime=1452788993030
2016-01-14 16:29:53,031 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788992843_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:53,031 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452788992843_0001_1_00 for vertex: vertex_1452788992843_0001_1_01 [C8139514E0A44FAAA5C3DE0F95F5617B] numStartedSources: 1 numSources: 1
2016-01-14 16:29:53,032 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788992843_0001_1_01 [C8139514E0A44FAAA5C3DE0F95F5617B] with distanceFromRoot: 1
2016-01-14 16:29:53,032 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788992843_0001_1_00 [FF732053E15E437C9E4FDB3854D9720A]
2016-01-14 16:29:53,032 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: FF732053E15E437C9E4FDB3854D9720A in vertex: C8139514E0A44FAAA5C3DE0F95F5617B
2016-01-14 16:29:53,033 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: C8139514E0A44FAAA5C3DE0F95F5617B with 1 source tasks and 1 pending tasks
2016-01-14 16:29:53,033 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:53,033 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992843_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788992843_0001_1_01, startRequestedTime=1452788993032, startedTime=1452788993032
2016-01-14 16:29:53,034 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788992843_0001_1_01 [C8139514E0A44FAAA5C3DE0F95F5617B] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:53,034 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [E8565EEA09A7493083CBF5CEA2006ACA/EE1EB61F3CF04BE7B2685A4D01379396] count/(1/1) ...latform/testCount2/count2, VertexName: FF732053E15E437C9E4FDB3854D9720A, VertexParallelism: 1, TaskAttemptID:attempt_1452788992843_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=89E2D24A5FB04F37A8345BDBAA483F71, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C8139514E0A44FAAA5C3DE0F95F5617B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 16:29:53,034 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788992843_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:53,034 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] submitted tez dag to app master: application_1452788992843_0001, with dag id: dag_1452788992843_0001_1
2016-01-14 16:29:53,035 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:53,036 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 16:29:53,036 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 16:29:53,037 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 16:29:53,037 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788992843_0001_00_000001
2016-01-14 16:29:53,037 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788992843_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:53,039 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788992843_0001_00_000001
2016-01-14 16:29:53,039 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 16:29:53,041 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:53,041 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992843_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788992843_0001_00_000001, launchTime=1452788993041
2016-01-14 16:29:53,042 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788992843_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:53,043 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788992843_0001_00_000001
2016-01-14 16:29:53,044 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788992843_0001_1_00_000000_0] to container: [container_1452788992843_0001_00_000001]
2016-01-14 16:29:53,044 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788992843_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:53,044 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788992843_0001_00_000001 given task: attempt_1452788992843_0001_1_00_000000_0
2016-01-14 16:29:53,044 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788992843_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788992843_0001_1_00_000000_0
2016-01-14 16:29:53,045 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788992843_0001_1_00_000000_0] started. Is using containerId: [container_1452788992843_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 16:29:53 Starting to run new task attempt: attempt_1452788992843_0001_1_00_000000_0
2016-01-14 16:29:53,045 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992843_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FF732053E15E437C9E4FDB3854D9720A, taskAttemptId=attempt_1452788992843_0001_1_00_000000_0, startTime=1452788993044, containerId=container_1452788992843_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788992843_0001_00_000001/teamcity, completedLogs=
2016-01-14 16:29:53,045 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788992843_0001_1_00_000000_0
2016-01-14 16:29:53,045 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:53,045 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:53,046 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:53,046 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:53,046 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:53,046 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:53,046 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:53,046 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E8565EEA09A7493083CBF5CEA2006ACA/EE1EB61F3CF04BE7B2685A4D01379396] count/(1/1) ...latform/testCount2/count2, VertexName: FF732053E15E437C9E4FDB3854D9720A, VertexParallelism: 1, TaskAttemptID:attempt_1452788992843_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=89E2D24A5FB04F37A8345BDBAA483F71, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C8139514E0A44FAAA5C3DE0F95F5617B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 16:29:53,046 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:53,047 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788992843_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 16:29:53,047 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788992843_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:53,047 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788992843_0001_1_00_000000_0
2016-01-14 16:29:53,048 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:53,048 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:53,048 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788992843_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:53,048 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:53,048 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C8139514E0A44FAAA5C3DE0F95F5617B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 16:29:53,049 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:53,049 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C8139514E0A44FAAA5C3DE0F95F5617B
2016-01-14 16:29:53,051 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 16:29:53,051 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C8139514E0A44FAAA5C3DE0F95F5617B
2016-01-14 16:29:53,052 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C8139514E0A44FAAA5C3DE0F95F5617B
2016-01-14 16:29:53,051 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=89E2D24A5FB04F37A8345BDBAA483F71, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 16:29:53,051 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:53,054 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:53,054 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 89E2D24A5FB04F37A8345BDBAA483F71
2016-01-14 16:29:53,054 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 89E2D24A5FB04F37A8345BDBAA483F71
2016-01-14 16:29:53,061 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 16:29:53,062 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 89E2D24A5FB04F37A8345BDBAA483F71
2016-01-14 16:29:53,062 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 89E2D24A5FB04F37A8345BDBAA483F71
2016-01-14 16:29:53,065 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FF732053E15E437C9E4FDB3854D9720A, ordinal: 0
2016-01-14 16:29:53,065 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF732053E15E437C9E4FDB3854D9720A, mem on start (mb), free: 164, total: 356, max: 672
2016-01-14 16:29:53,065 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:53,065 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:53,065 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:53,066 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:53,066 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:53,066 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 16:29:53,067 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 16:29:53,067 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:53,067 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 16:29:53,067 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 16:29:53,068 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C8139514E0A44FAAA5C3DE0F95F5617B, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 16:29:53,068 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 89E2D24A5FB04F37A8345BDBAA483F71, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 16:29:53,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:53,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 89E2D24A5FB04F37A8345BDBAA483F71 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:53,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 16:29:53,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:53,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 89E2D24A5FB04F37A8345BDBAA483F71
2016-01-14 16:29:53,070 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:53,070 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 89E2D24A5FB04F37A8345BDBAA483F71
2016-01-14 16:29:53,070 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:53,071 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788992843_0001_1_00_000000_0
2016-01-14 16:29:53,074 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 16:29:53,074 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: 89E2D24A5FB04F37A8345BDBAA483F71
2016-01-14 16:29:53,075 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: 740E8CF28F97412B92C47BDD298865DC
2016-01-14 16:29:53,075 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] 740E8CF28F97412B92C47BDD298865DC
2016-01-14 16:29:53,075 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 16:29:53,075 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@75eff472; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@5c99a4db; comparator=cascading.tuple.hadoop.util.TupleComparator@6b16555b; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 16:29:53,076 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@73e004a
2016-01-14 16:29:53,076 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 16:29:53,078 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 16:29:53,078 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 16:29:53,078 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@73e004a
2016-01-14 16:29:53,088 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 16:29:53,089 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 16:29:53,100 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 16:29:53,100 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 16:29:53,100 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 16:29:53,100 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 16:29:53,101 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 16:29:53,102 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 16:29:53,102 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:53,102 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FF732053E15E437C9E4FDB3854D9720A, all 1 inputs ready in: 00:00:00.000
2016-01-14 16:29:53,107 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788992843_0001_1_00_000000_0
2016-01-14 16:29:53,107 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF732053E15E437C9E4FDB3854D9720A, mem on close (mb), free: 222, total: 357, max: 672
2016-01-14 16:29:53,107 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 16:29:53,107 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 16:29:53,107 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2016-01-14 16:29:53,107 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 16:29:53,111 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 16:29:53 Completed running task attempt: attempt_1452788992843_0001_1_00_000000_0
2016-01-14 16:29:53,117 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788992843_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:53,119 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992843_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788992843_0001_1_00, eventCount=1
2016-01-14 16:29:53,119 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:53,119 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992843_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FF732053E15E437C9E4FDB3854D9720A, taskAttemptId=attempt_1452788992843_0001_1_00_000000_0, startTime=1452788993044, finishTime=1452788993119, timeTaken=75, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788993102, Process_Duration=5, Process_End_Time=1452788993107, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 16:29:53,120 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788992843_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:53,120 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788992843_0001_1_00_000000_0
2016-01-14 16:29:53,120 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:53,121 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992843_0001_1][Event:TASK_FINISHED]: vertexName=FF732053E15E437C9E4FDB3854D9720A, taskId=task_1452788992843_0001_1_00_000000, startTime=1452788993044, finishTime=1452788993120, timeTaken=76, status=SUCCEEDED, successfulAttemptID=attempt_1452788992843_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788993102, Process_Duration=5, Process_End_Time=1452788993107, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 16:29:53,121 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788992843_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:53,121 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788992843_0001_1_00 [FF732053E15E437C9E4FDB3854D9720A] : 1
2016-01-14 16:29:53,121 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788992843_0001_1_00 [FF732053E15E437C9E4FDB3854D9720A]
2016-01-14 16:29:53,119 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452788993102, Process_Duration=5, Process_End_Time=1452788993107, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 16:29:53,122 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992843_0001_1][Event:VERTEX_FINISHED]: vertexName=FF732053E15E437C9E4FDB3854D9720A, vertexId=vertex_1452788992843_0001_1_00, initRequestedTime=1452788993015, initedTime=1452788993028, startRequestedTime=1452788993029, startedTime=1452788993029, finishTime=1452788993121, timeTaken=92, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788993102, Process_Duration=5, Process_End_Time=1452788993107, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452788993044, firstTasksToStart=[ task_1452788992843_0001_1_00_000000 ], lastTaskFinishTime=1452788993119, lastTasksToFinish=[ task_1452788992843_0001_1_00_000000 ], minTaskDuration=75, maxTaskDuration=75, avgTaskDuration=75.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788992843_0001_1_00_000000 ], longestDurationTasks=[ task_1452788992843_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:53,123 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:53,123 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788992843_0001_1_00 [FF732053E15E437C9E4FDB3854D9720A] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:53,122 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:53,120 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:53,124 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:53,124 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788992843_0001_00_000001
2016-01-14 16:29:53,125 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452788992843_0001_1_01 [C8139514E0A44FAAA5C3DE0F95F5617B] attempt: attempt_1452788992843_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 16:29:53,125 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: C8139514E0A44FAAA5C3DE0F95F5617B
2016-01-14 16:29:53,125 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788992843_0001_1_01 [C8139514E0A44FAAA5C3DE0F95F5617B]
2016-01-14 16:29:53,125 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788992843_0001_1_00 [FF732053E15E437C9E4FDB3854D9720A] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 16:29:53,125 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788992843_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:53,126 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992843_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788992843_0001_00_000001, stoppedTime=1452788993125, exitStatus=0
2016-01-14 16:29:53,126 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788992843_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:53,126 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992843_0001_1][Event:TASK_STARTED]: vertexName=C8139514E0A44FAAA5C3DE0F95F5617B, taskId=task_1452788992843_0001_1_01_000000, scheduledTime=1452788993126, launchTime=1452788993126
2016-01-14 16:29:53,126 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788992843_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:53,126 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788992843_0001_1_01 [C8139514E0A44FAAA5C3DE0F95F5617B]
2016-01-14 16:29:53,127 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [E8565EEA09A7493083CBF5CEA2006ACA/EE1EB61F3CF04BE7B2685A4D01379396] count/(1/1) ...latform/testCount2/count2, VertexName: C8139514E0A44FAAA5C3DE0F95F5617B, VertexParallelism: 1, TaskAttemptID:attempt_1452788992843_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FF732053E15E437C9E4FDB3854D9720A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=5EEB1893C29648919A77CA139100726A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:53,127 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788992843_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:53,127 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:53,127 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452788992843_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 16:29:53,127 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788992843_0001_00_000002
2016-01-14 16:29:53,128 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788992843_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:53,128 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788992843_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 16:29:53,129 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788992843_0001_00_000002
2016-01-14 16:29:53,129 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 16:29:53,126 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:53,130 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:53,132 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788992843_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:53,132 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:53,132 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992843_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788992843_0001_00_000002, launchTime=1452788993132
2016-01-14 16:29:53,132 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788992843_0001_00_000002
2016-01-14 16:29:53,133 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788992843_0001_1_01_000000_0] to container: [container_1452788992843_0001_00_000002]
2016-01-14 16:29:53,133 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788992843_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:53,133 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788992843_0001_00_000002 given task: attempt_1452788992843_0001_1_01_000000_0
2016-01-14 16:29:53,133 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788992843_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788992843_0001_1_01_000000_0
2016-01-14 16:29:53,134 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788992843_0001_1_01_000000_0] started. Is using containerId: [container_1452788992843_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 16:29:53,134 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992843_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C8139514E0A44FAAA5C3DE0F95F5617B, taskAttemptId=attempt_1452788992843_0001_1_01_000000_0, startTime=1452788993133, containerId=container_1452788992843_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788992843_0001_00_000002/teamcity, completedLogs=
2016-01-14 16:29:53 Starting to run new task attempt: attempt_1452788992843_0001_1_01_000000_0
2016-01-14 16:29:53,134 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:53,134 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788992843_0001_1_01_000000_0
2016-01-14 16:29:53,134 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788992843_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:53,134 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788992843_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:53,134 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:53,135 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:53,135 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:53,135 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:53,135 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:53,135 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:53,136 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E8565EEA09A7493083CBF5CEA2006ACA/EE1EB61F3CF04BE7B2685A4D01379396] count/(1/1) ...latform/testCount2/count2, VertexName: C8139514E0A44FAAA5C3DE0F95F5617B, VertexParallelism: 1, TaskAttemptID:attempt_1452788992843_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FF732053E15E437C9E4FDB3854D9720A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=5EEB1893C29648919A77CA139100726A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:53,136 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:53,137 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788992843_0001_1_01_000000_0
2016-01-14 16:29:53,137 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:53,137 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:53,138 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452788992843_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 16:29:53,139 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788992843_0001_00_000001: 14 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 16:29:53,139 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452788992843_0001_00_000001, Exiting
2016-01-14 16:29:53,139 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788992843_0001_00_000001
2016-01-14 16:29:53,139 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:53,139 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=5EEB1893C29648919A77CA139100726A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 16:29:53,139 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:53,140 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 5EEB1893C29648919A77CA139100726A
2016-01-14 16:29:53,140 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 16:29:53,140 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 5EEB1893C29648919A77CA139100726A
2016-01-14 16:29:53,138 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=FF732053E15E437C9E4FDB3854D9720A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 16:29:53,139 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788992843_0001_00_000001
2016-01-14 16:29:53,139 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788992843_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 16:29:53,141 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:53,141 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: FF732053E15E437C9E4FDB3854D9720A
2016-01-14 16:29:53,142 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:53,146 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788992843_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 16:29:53,146 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452788992843_0001_00_000001 completed successfully
2016-01-14 16:29:53,147 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788992843_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 16:29:53,147 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 16:29:53,147 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 16:29:53,147 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: FF732053E15E437C9E4FDB3854D9720A
2016-01-14 16:29:53,148 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: FF732053E15E437C9E4FDB3854D9720A
2016-01-14 16:29:53,156 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 16:29:53,156 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 5EEB1893C29648919A77CA139100726A
2016-01-14 16:29:53,157 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C8139514E0A44FAAA5C3DE0F95F5617B, ordinal: 1
2016-01-14 16:29:53,157 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C8139514E0A44FAAA5C3DE0F95F5617B, mem on start (mb), free: 215, total: 357, max: 672
2016-01-14 16:29:53,157 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:53,157 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:53,158 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:53,158 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:53,158 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:53,159 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 16:29:53,159 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:53,159 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 16:29:53,160 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 16:29:53,160 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 16:29:53,160 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 5EEB1893C29648919A77CA139100726A, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 16:29:53,160 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, FF732053E15E437C9E4FDB3854D9720A, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 16:29:53,161 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:53,161 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: FF732053E15E437C9E4FDB3854D9720A
2016-01-14 16:29:53,162 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 16:29:53,162 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 16:29:53,163 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: FF732053E15E437C9E4FDB3854D9720A being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:53,163 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: FF732053E15E437C9E4FDB3854D9720A with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 16:29:53,163 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 16:29:53,163 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 16:29:53,164 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 16:29:53,165 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: FF732053E15E437C9E4FDB3854D9720A
2016-01-14 16:29:53,165 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 16:29:53,169 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:53,169 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:53,169 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788992843_0001_1_01_000000_0
2016-01-14 16:29:53,173 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452788992843_0001_1_00_000000_0_10067, runDuration: 65178, ]
2016-01-14 16:29:53,173 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-01-14 16:29:53,174 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: 740E8CF28F97412B92C47BDD298865DC
2016-01-14 16:29:53,174 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2"], id: 5EEB1893C29648919A77CA139100726A
2016-01-14 16:29:53,174 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2"]
2016-01-14 16:29:53,175 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] 740E8CF28F97412B92C47BDD298865DC, for 1 inputs
2016-01-14 16:29:53,176 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452788992843_0001_1_00_000000_0_10067] to DISK_DIRECT, CompressedSize=463, DecompressedSize=459,EndTime=1452788993176, TimeTaken=1, Rate=0.44 MB/s
2016-01-14 16:29:53,176 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : FF732053E15E437C9E4FDB3854D9720A
2016-01-14 16:29:53,176 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 16:29:53,177 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [FF732053E15E437C9E4FDB3854D9720A] #11 in 2ms
2016-01-14 16:29:53,177 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [FF732053E15E437C9E4FDB3854D9720A] #11, status:false, isInterrupted:false
2016-01-14 16:29:53,178 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 16:29:53,178 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 463 bytes from disk
2016-01-14 16:29:53,178 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 16:29:53,178 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 16:29:53,178 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-01-14 16:29:53,179 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : FF732053E15E437C9E4FDB3854D9720A
2016-01-14 16:29:53,179 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 16:29:53,179 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C8139514E0A44FAAA5C3DE0F95F5617B, all 1 inputs ready in: 00:00:00.004
2016-01-14 16:29:53,182 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3ce928bf; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@6f7232d6
2016-01-14 16:29:53,186 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 16:29:53,187 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 16:29:53,187 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452788992843_0001_1_01_000000_0
2016-01-14 16:29:53,187 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452788992843_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 16:29:53,189 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788992843_0001_1_01_000000_0
2016-01-14 16:29:53,189 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C8139514E0A44FAAA5C3DE0F95F5617B, mem on close (mb), free: 213, total: 357, max: 672
2016-01-14 16:29:53,189 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: FF732053E15E437C9E4FDB3854D9720A
2016-01-14 16:29:53,190 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 16:29:53,190 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 16:29:53,190 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 16:29:53,190 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 16:29:53,191 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 16:29:53 Completed running task attempt: attempt_1452788992843_0001_1_01_000000_0
2016-01-14 16:29:53,192 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788992843_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:53,193 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=15, MERGE_PHASE_TIME=17, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452788993180, Process_Duration=9, Process_End_Time=1452788993189, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=17]]
2016-01-14 16:29:53,193 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:53,194 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992843_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C8139514E0A44FAAA5C3DE0F95F5617B, taskAttemptId=attempt_1452788992843_0001_1_01_000000_0, startTime=1452788993133, finishTime=1452788993193, timeTaken=60, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=15, MERGE_PHASE_TIME=17, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788993180, Process_Duration=9, Process_End_Time=1452788993189, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-01-14 16:29:53,194 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788992843_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:53,194 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788992843_0001_1_01_000000_0
2016-01-14 16:29:53,195 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:53,195 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:53,195 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992843_0001_1][Event:TASK_FINISHED]: vertexName=C8139514E0A44FAAA5C3DE0F95F5617B, taskId=task_1452788992843_0001_1_01_000000, startTime=1452788993133, finishTime=1452788993194, timeTaken=61, status=SUCCEEDED, successfulAttemptID=attempt_1452788992843_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=15, MERGE_PHASE_TIME=17, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788993180, Process_Duration=9, Process_End_Time=1452788993189, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-01-14 16:29:53,195 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788992843_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:53,195 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788992843_0001_1_01 [C8139514E0A44FAAA5C3DE0F95F5617B] : 1
2016-01-14 16:29:53,195 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788992843_0001_1_01 [C8139514E0A44FAAA5C3DE0F95F5617B]
2016-01-14 16:29:53,195 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:53,197 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992843_0001_1][Event:VERTEX_FINISHED]: vertexName=C8139514E0A44FAAA5C3DE0F95F5617B, vertexId=vertex_1452788992843_0001_1_01, initRequestedTime=1452788993016, initedTime=1452788993025, startRequestedTime=1452788993032, startedTime=1452788993032, finishTime=1452788993195, timeTaken=163, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=15, MERGE_PHASE_TIME=17, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788993180, Process_Duration=9, Process_End_Time=1452788993189, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1452788993133, firstTasksToStart=[ task_1452788992843_0001_1_01_000000 ], lastTaskFinishTime=1452788993193, lastTasksToFinish=[ task_1452788992843_0001_1_01_000000 ], minTaskDuration=60, maxTaskDuration=60, avgTaskDuration=60.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788992843_0001_1_01_000000 ], longestDurationTasks=[ task_1452788992843_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:53,197 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788992843_0001_1_01 [C8139514E0A44FAAA5C3DE0F95F5617B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:53,197 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788992843_0001_1_01 [C8139514E0A44FAAA5C3DE0F95F5617B] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 16:29:53,197 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452788992843_0001_1
2016-01-14 16:29:53,197 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992843_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452788992843_0001_1
2016-01-14 16:29:53,197 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452788992843_0001_1_00 [FF732053E15E437C9E4FDB3854D9720A]
2016-01-14 16:29:53,197 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 5EEB1893C29648919A77CA139100726A for vertex: vertex_1452788992843_0001_1_01 [C8139514E0A44FAAA5C3DE0F95F5617B]
2016-01-14 16:29:53,195 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788992843_0001_00_000002
2016-01-14 16:29:53,197 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:53,195 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:53,208 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452788992843_0001_1, queueSize=0
2016-01-14 16:29:53,208 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 16:29:53,209 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992843_0001_1][Event:DAG_FINISHED]: dagId=dag_1452788992843_0001_1, startTime=1452788993014, finishTime=1452788993206, timeTaken=192, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=940, FILE_BYTES_WRITTEN=1297, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=749731840, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=15, MERGE_PHASE_TIME=17, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905577986282, Process_Duration=14, Process_End_Time=2905577986296, Read_Duration=5, Tuples_Read=40, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-01-14 16:29:53,209 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452788992843_0001_1 finished with state: SUCCEEDED
2016-01-14 16:29:53,210 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788992843_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 16:29:53,210 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788992843_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:53,211 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:53,211 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788992843_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788992843_0001_00_000002, stoppedTime=1452788993210, exitStatus=0
2016-01-14 16:29:53,211 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788992843_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:53 Completed Dag: dag_1452788992843_0001_1
2016-01-14 16:29:53,212 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452788992843_0001_1, dagState=SUCCEEDED
2016-01-14 16:29:53,212 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788992843_0001_1_post
2016-01-14 16:29:53,213 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:53,213 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 16:29:53,213 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452788992843_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 16:29:53,214 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788992843_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 16:29:53,218 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testCount2/count2, applicationId=application_1452788992843_0001
2016-01-14 16:29:53,222 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 16:29:53,222 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 16:29:53,222 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 16:29:53,222 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 16:29:53,222 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 16:29:53,223 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 16:29:53,223 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 16:29:53,223 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 16:29:53,224 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 16:29:53,224 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 16:29:53,224 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 16:29:53,225 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 16:29:53,225 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 16:29:53,225 INFO task.TezChild (TezChild.java:run(198)) - Interrupted while waiting for new work for container container_1452788992843_0001_00_000002
2016-01-14 16:29:53,226 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788992843_0001_00_000002
2016-01-14 16:29:53,226 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788992843_0001_00_000002
2016-01-14 16:29:53,226 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788992843_0001_00_000002 = ContainerExecutionResult{exitStatus=INTERRUPTED, throwable=java.lang.InterruptedException, errorMessage='Interrupted while waiting for new work'}
2016-01-14 16:29:53,226 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452788992843_0001_00_000002 completed but with errors
2016-01-14 16:29:53,226 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452788992843_0001_00_000002 exited with diagnostics set to Interrupted while waiting for new work
2016-01-14 16:29:53,227 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788992843_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 16:29:53,230 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E8565EEA09A7493083CBF5CEA2006ACA/EE1EB61F3CF04BE7B2685A4D01379396/.tez/application_1452788992843_0001
2016-01-14 16:29:53,231 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] completed in: 00:00.407
2016-01-14 16:29:53,302 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount2/count2
2016-01-14 16:29:53,323 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 16:29:53,324 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 16:29:53,359 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-01-14 16:29:53,372 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.025
2016-01-14 16:29:53,373 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 16:29:53,373 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 16:29:53,373 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 16:29:53,396 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-01-14 16:29:53,404 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3
2016-01-14 16:29:53,409 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 16:29:53,409 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 16:29:53,456 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] starting
2016-01-14 16:29:53,456 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 16:29:53,456 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3"]
2016-01-14 16:29:53,456 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] parallel execution of steps is enabled: false
2016-01-14 16:29:53,456 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executing total steps: 1
2016-01-14 16:29:53,456 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] allocating management threads: 1
2016-01-14 16:29:53,457 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] starting step: (1/1) ...latform/testCount3/count3
2016-01-14 16:29:53,457 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 16:29:53,469 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tez session mode enabled: true
2016-01-14 16:29:53,474 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:53,474 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 16:29:53,474 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 16:29:53,483 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C821BCDF18DA42D68A88137F9D6F06E9/EDF6AE7F29DD491CA7C7CF7DE020CAF1/.tez/application_1452788993474_0001 doesn't exist and is created
2016-01-14 16:29:53,499 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 16:29:53,499 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 16:29:53,500 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C821BCDF18DA42D68A88137F9D6F06E9/EDF6AE7F29DD491CA7C7CF7DE020CAF1/.tez/application_1452788993474_0001
2016-01-14 16:29:53,503 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452788993474_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:53,513 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 16:29:53,514 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 16:29:53,515 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 16:29:53,516 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 16:29:53,516 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C821BCDF18DA42D68A88137F9D6F06E9/EDF6AE7F29DD491CA7C7CF7DE020CAF1/.tez/application_1452788993474_0001/localmode-log-dir
2016-01-14 16:29:53,516 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C821BCDF18DA42D68A88137F9D6F06E9/EDF6AE7F29DD491CA7C7CF7DE020CAF1/.tez/application_1452788993474_0001/localmode-log-dir/history.txt.appattempt_1452788993474_0001_000000, maxErrors=10
2016-01-14 16:29:53,516 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 16:29:53,516 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452788993474_0001_000000, appSubmitTime=1452788993503, launchTime=1452788993503
2016-01-14 16:29:53,520 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 16:29:53,523 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 16:29:53,524 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 16:29:53,524 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 16:29:53,529 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:36616
2016-01-14 16:29:53,530 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 16:29:53,533 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452788993474_0001_000000, startTime=1452788993533
2016-01-14 16:29:53,533 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 16:29:53,533 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 16:29:53,599 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 16:29:53,600 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 16:29:53,600 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testCount3/count3, applicationId=application_1452788993474_0001, dagName=[C821BCDF18DA42D68A88137F9D6F06E9/EDF6AE7F29DD491CA7C7CF7DE020CAF1] count/(1/1) ...latform/testCount3/count3
2016-01-14 16:29:53,615 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [C821BCDF18DA42D68A88137F9D6F06E9/EDF6AE7F29DD491CA7C7CF7DE020CAF1] count/(1/1) ...latform/testCount3/count3
2016-01-14 16:29:53,617 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452788993474_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C821BCDF18DA42D68A88137F9D6F06E9/EDF6AE7F29DD491CA7C7CF7DE020CAF1/.tez/application_1452788993474_0001/localmode-log-dir/dag_1452788993474_0001_1.dot
2016-01-14 16:29:53,617 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C821BCDF18DA42D68A88137F9D6F06E9/EDF6AE7F29DD491CA7C7CF7DE020CAF1/.tez/application_1452788993474_0001/localmode-log-dir/dag_1452788993474_0001_1-tez-dag.pb.txt
2016-01-14 16:29:53,625 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788993474_0001_1
2016-01-14 16:29:53,625 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:53,625 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [C821BCDF18DA42D68A88137F9D6F06E9/EDF6AE7F29DD491CA7C7CF7DE020CAF1] count/(1/1) ...latform/testCount3/count3
2016-01-14 16:29:53 Running Dag: dag_1452788993474_0001_1
2016-01-14 16:29:53,641 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788993474_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452788993474_0001_1, submitTime=1452788993616
2016-01-14 16:29:53,642 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E515477DC8B447F0B9913E106386FC38
2016-01-14 16:29:53,642 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 16:29:53,642 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788993474_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452788993474_0001_1, initTime=1452788993641
2016-01-14 16:29:53,642 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788993474_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 16:29:53,643 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 16:29:53,643 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 16:29:53,644 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788993474_0001_1][Event:DAG_STARTED]: dagID=dag_1452788993474_0001_1, startTime=1452788993644
2016-01-14 16:29:53,644 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 16:29:53,644 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788993474_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 16:29:53,644 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: EABB36831C1045E1BA44BA3554FB5325 : {FF9AC475467F45AEB8A6FE291C96FF9D={InputName=FF9AC475467F45AEB8A6FE291C96FF9D}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 16:29:53,645 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: FF9AC475467F45AEB8A6FE291C96FF9D, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 16:29:53,645 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452788993474_0001_1_00 [EABB36831C1045E1BA44BA3554FB5325]
2016-01-14 16:29:53,645 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452788993474_0001_1_00 [EABB36831C1045E1BA44BA3554FB5325]
2016-01-14 16:29:53,645 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452788993474_0001_1_00 [EABB36831C1045E1BA44BA3554FB5325]
2016-01-14 16:29:53,645 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452788993474_0001_1_00 [EABB36831C1045E1BA44BA3554FB5325]. Starting root input initializers: 1
2016-01-14 16:29:53,646 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 16:29:53,646 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788993474_0001_1_00 [EABB36831C1045E1BA44BA3554FB5325] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 16:29:53,646 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452788993474_0001_1_01 [E515477DC8B447F0B9913E106386FC38]
2016-01-14 16:29:53,649 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 16:29:53,649 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: FF9AC475467F45AEB8A6FE291C96FF9D on vertex vertex_1452788993474_0001_1_00 [EABB36831C1045E1BA44BA3554FB5325]
2016-01-14 16:29:53,651 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testCount3/count3, applicationId=application_1452788993474_0001, dagName=[C821BCDF18DA42D68A88137F9D6F06E9/EDF6AE7F29DD491CA7C7CF7DE020CAF1] count/(1/1) ...latform/testCount3/count3
2016-01-14 16:29:53,649 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452788993474_0001_1_01 [E515477DC8B447F0B9913E106386FC38]
2016-01-14 16:29:53,652 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452788993474_0001_1_01 [E515477DC8B447F0B9913E106386FC38]
2016-01-14 16:29:53,652 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452788993474_0001_1_01 [E515477DC8B447F0B9913E106386FC38]
2016-01-14 16:29:53,652 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=A5F3CCDE8D304B74A7A03CC6306B8C29, vertexId=vertex_1452788993474_0001_1_01 [E515477DC8B447F0B9913E106386FC38], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 16:29:53,652 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=A5F3CCDE8D304B74A7A03CC6306B8C29, vertexId=vertex_1452788993474_0001_1_01 [E515477DC8B447F0B9913E106386FC38]
2016-01-14 16:29:53,654 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=A5F3CCDE8D304B74A7A03CC6306B8C29, vertexName=E515477DC8B447F0B9913E106386FC38, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 16:29:53,654 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=A5F3CCDE8D304B74A7A03CC6306B8C29, vertexId=vertex_1452788993474_0001_1_01 [E515477DC8B447F0B9913E106386FC38]
2016-01-14 16:29:53,656 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 16:29:53,656 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:53,656 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input FF9AC475467F45AEB8A6FE291C96FF9D asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 16:29:53,656 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788993474_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E515477DC8B447F0B9913E106386FC38, vertexId=vertex_1452788993474_0001_1_01, initRequestedTime=1452788993646, initedTime=1452788993656, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 16:29:53,656 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788993474_0001_1_01 [E515477DC8B447F0B9913E106386FC38] transitioned from NEW to INITED due to event V_INIT
2016-01-14 16:29:53,656 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 16:29:53,657 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 16:29:53,659 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-01-14 16:29:53,659 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: FF9AC475467F45AEB8A6FE291C96FF9D on vertex vertex_1452788993474_0001_1_00 [EABB36831C1045E1BA44BA3554FB5325]
2016-01-14 16:29:53,659 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788993474_0001_1_00 [EABB36831C1045E1BA44BA3554FB5325] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:53,660 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {FF9AC475467F45AEB8A6FE291C96FF9D=forAllWorkUnits=true, update=[1]}
2016-01-14 16:29:53,660 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452788993474_0001_1_00 [EABB36831C1045E1BA44BA3554FB5325] parallelism set to 1
2016-01-14 16:29:53,660 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788993474_0001_1_00 [EABB36831C1045E1BA44BA3554FB5325] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:53,661 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788993474_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788993474_0001_1_00, eventCount=1
2016-01-14 16:29:53,662 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:53,662 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788993474_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EABB36831C1045E1BA44BA3554FB5325, vertexId=vertex_1452788993474_0001_1_00, initRequestedTime=1452788993644, initedTime=1452788993661, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 16:29:53,662 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788993474_0001_1_00 [EABB36831C1045E1BA44BA3554FB5325] with distanceFromRoot: 0
2016-01-14 16:29:53,663 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788993474_0001_1_00 [EABB36831C1045E1BA44BA3554FB5325] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 16:29:53,663 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in EABB36831C1045E1BA44BA3554FB5325
2016-01-14 16:29:53,663 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788993474_0001_1_00 [EABB36831C1045E1BA44BA3554FB5325]
2016-01-14 16:29:53,664 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:53,664 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788993474_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788993474_0001_1_00, startRequestedTime=1452788993663, startedTime=1452788993663
2016-01-14 16:29:53,664 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788993474_0001_1_00 [EABB36831C1045E1BA44BA3554FB5325] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:53,665 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:53,665 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788993474_0001_1][Event:TASK_STARTED]: vertexName=EABB36831C1045E1BA44BA3554FB5325, taskId=task_1452788993474_0001_1_00_000000, scheduledTime=1452788993665, launchTime=1452788993665
2016-01-14 16:29:53,665 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788993474_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:53,666 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452788993474_0001_1_00 for vertex: vertex_1452788993474_0001_1_01 [E515477DC8B447F0B9913E106386FC38] numStartedSources: 1 numSources: 1
2016-01-14 16:29:53,666 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788993474_0001_1_01 [E515477DC8B447F0B9913E106386FC38] with distanceFromRoot: 1
2016-01-14 16:29:53,666 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788993474_0001_1_00 [EABB36831C1045E1BA44BA3554FB5325]
2016-01-14 16:29:53,667 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: EABB36831C1045E1BA44BA3554FB5325 in vertex: E515477DC8B447F0B9913E106386FC38
2016-01-14 16:29:53,667 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E515477DC8B447F0B9913E106386FC38 with 1 source tasks and 1 pending tasks
2016-01-14 16:29:53,667 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:53,668 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788993474_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788993474_0001_1_01, startRequestedTime=1452788993666, startedTime=1452788993666
2016-01-14 16:29:53,668 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788993474_0001_1_01 [E515477DC8B447F0B9913E106386FC38] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:53,668 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C821BCDF18DA42D68A88137F9D6F06E9/EDF6AE7F29DD491CA7C7CF7DE020CAF1] count/(1/1) ...latform/testCount3/count3, VertexName: EABB36831C1045E1BA44BA3554FB5325, VertexParallelism: 1, TaskAttemptID:attempt_1452788993474_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FF9AC475467F45AEB8A6FE291C96FF9D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E515477DC8B447F0B9913E106386FC38, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 16:29:53,668 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788993474_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:53,669 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:53,670 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 16:29:53,670 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] submitted tez dag to app master: application_1452788993474_0001, with dag id: dag_1452788993474_0001_1
2016-01-14 16:29:53,670 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 16:29:53,671 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 16:29:53,671 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788993474_0001_00_000001
2016-01-14 16:29:53,671 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788993474_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:53,673 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788993474_0001_00_000001
2016-01-14 16:29:53,674 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 16:29:53,674 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788993474_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:53,674 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:53,675 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788993474_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788993474_0001_00_000001, launchTime=1452788993674
2016-01-14 16:29:53,676 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788993474_0001_00_000001
2016-01-14 16:29:53,677 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788993474_0001_1_00_000000_0] to container: [container_1452788993474_0001_00_000001]
2016-01-14 16:29:53,678 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788993474_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:53,678 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788993474_0001_00_000001 given task: attempt_1452788993474_0001_1_00_000000_0
2016-01-14 16:29:53,678 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788993474_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788993474_0001_1_00_000000_0
2016-01-14 16:29:53,678 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788993474_0001_1_00_000000_0] started. Is using containerId: [container_1452788993474_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 16:29:53,679 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788993474_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EABB36831C1045E1BA44BA3554FB5325, taskAttemptId=attempt_1452788993474_0001_1_00_000000_0, startTime=1452788993678, containerId=container_1452788993474_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788993474_0001_00_000001/teamcity, completedLogs=
2016-01-14 16:29:53 Starting to run new task attempt: attempt_1452788993474_0001_1_00_000000_0
2016-01-14 16:29:53,679 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788993474_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:53,679 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:53,679 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788993474_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:53,679 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788993474_0001_1_00_000000_0
2016-01-14 16:29:53,680 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:53,680 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:53,680 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:53,680 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:53,680 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:53,680 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:53,680 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C821BCDF18DA42D68A88137F9D6F06E9/EDF6AE7F29DD491CA7C7CF7DE020CAF1] count/(1/1) ...latform/testCount3/count3, VertexName: EABB36831C1045E1BA44BA3554FB5325, VertexParallelism: 1, TaskAttemptID:attempt_1452788993474_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FF9AC475467F45AEB8A6FE291C96FF9D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E515477DC8B447F0B9913E106386FC38, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 16:29:53,681 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:53,684 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788993474_0001_1_00_000000_0
2016-01-14 16:29:53,685 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:53,685 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:53,685 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788993474_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 16:29:53,686 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=FF9AC475467F45AEB8A6FE291C96FF9D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 16:29:53,686 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E515477DC8B447F0B9913E106386FC38, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 16:29:53,686 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:53,687 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E515477DC8B447F0B9913E106386FC38
2016-01-14 16:29:53,686 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:53,686 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:53,687 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: FF9AC475467F45AEB8A6FE291C96FF9D
2016-01-14 16:29:53,688 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: FF9AC475467F45AEB8A6FE291C96FF9D
2016-01-14 16:29:53,689 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 16:29:53,689 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E515477DC8B447F0B9913E106386FC38
2016-01-14 16:29:53,690 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E515477DC8B447F0B9913E106386FC38
2016-01-14 16:29:53,692 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 16:29:53,692 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: FF9AC475467F45AEB8A6FE291C96FF9D
2016-01-14 16:29:53,692 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: FF9AC475467F45AEB8A6FE291C96FF9D
2016-01-14 16:29:53,692 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:53,698 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EABB36831C1045E1BA44BA3554FB5325, ordinal: 0
2016-01-14 16:29:53,698 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EABB36831C1045E1BA44BA3554FB5325, mem on start (mb), free: 123, total: 357, max: 672
2016-01-14 16:29:53,698 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:53,698 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:53,699 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:53,699 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:53,699 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:53,699 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 16:29:53,699 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:53,699 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 16:29:53,700 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 16:29:53,700 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 16:29:53,700 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, FF9AC475467F45AEB8A6FE291C96FF9D, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 16:29:53,700 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E515477DC8B447F0B9913E106386FC38, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 16:29:53,700 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:53,700 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: FF9AC475467F45AEB8A6FE291C96FF9D being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:53,701 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 16:29:53,701 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:53,701 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: FF9AC475467F45AEB8A6FE291C96FF9D
2016-01-14 16:29:53,701 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:53,701 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: FF9AC475467F45AEB8A6FE291C96FF9D
2016-01-14 16:29:53,701 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:53,702 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788993474_0001_1_00_000000_0
2016-01-14 16:29:53,704 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 16:29:53,705 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@385bbe
2016-01-14 16:29:53,705 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 16:29:53,705 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: FF9AC475467F45AEB8A6FE291C96FF9D
2016-01-14 16:29:53,705 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@385bbe
2016-01-14 16:29:53,706 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 16:29:53,706 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 16:29:53,706 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: 05E300758FA2457BBCACA86C9BC79B9A
2016-01-14 16:29:53,706 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] 05E300758FA2457BBCACA86C9BC79B9A
2016-01-14 16:29:53,707 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@6daf6f1c; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@18a7cf86; comparator=cascading.tuple.hadoop.util.TupleComparator@3129730f; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 16:29:53,708 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 16:29:53,708 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 16:29:53,708 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 16:29:53,731 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 16:29:53,732 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 16:29:53,732 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 16:29:53,732 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 16:29:53,733 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 16:29:53,733 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 16:29:53,734 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:53,734 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EABB36831C1045E1BA44BA3554FB5325, all 1 inputs ready in: 00:00:00.000
2016-01-14 16:29:53,737 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788993474_0001_1_00_000000_0
2016-01-14 16:29:53,737 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EABB36831C1045E1BA44BA3554FB5325, mem on close (mb), free: 220, total: 356, max: 672
2016-01-14 16:29:53,737 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 16:29:53,737 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 16:29:53,737 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2016-01-14 16:29:53,738 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 16:29:53,741 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 16:29:53 Completed running task attempt: attempt_1452788993474_0001_1_00_000000_0
2016-01-14 16:29:53,745 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788993474_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:53,745 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788993474_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788993474_0001_1_00, eventCount=1
2016-01-14 16:29:53,746 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:53,746 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788993474_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EABB36831C1045E1BA44BA3554FB5325, taskAttemptId=attempt_1452788993474_0001_1_00_000000_0, startTime=1452788993678, finishTime=1452788993746, timeTaken=68, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788993734, Process_Duration=3, Process_End_Time=1452788993737, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 16:29:53,746 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788993474_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:53,746 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788993474_0001_1_00_000000_0
2016-01-14 16:29:53,747 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:53,747 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788993474_0001_1][Event:TASK_FINISHED]: vertexName=EABB36831C1045E1BA44BA3554FB5325, taskId=task_1452788993474_0001_1_00_000000, startTime=1452788993678, finishTime=1452788993747, timeTaken=69, status=SUCCEEDED, successfulAttemptID=attempt_1452788993474_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788993734, Process_Duration=3, Process_End_Time=1452788993737, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 16:29:53,747 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788993474_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:53,747 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:53,747 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788993474_0001_1_00 [EABB36831C1045E1BA44BA3554FB5325] : 1
2016-01-14 16:29:53,748 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788993474_0001_1_00 [EABB36831C1045E1BA44BA3554FB5325]
2016-01-14 16:29:53,750 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:53,750 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788993474_0001_1][Event:VERTEX_FINISHED]: vertexName=EABB36831C1045E1BA44BA3554FB5325, vertexId=vertex_1452788993474_0001_1_00, initRequestedTime=1452788993644, initedTime=1452788993661, startRequestedTime=1452788993663, startedTime=1452788993663, finishTime=1452788993748, timeTaken=85, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788993734, Process_Duration=3, Process_End_Time=1452788993737, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452788993678, firstTasksToStart=[ task_1452788993474_0001_1_00_000000 ], lastTaskFinishTime=1452788993746, lastTasksToFinish=[ task_1452788993474_0001_1_00_000000 ], minTaskDuration=68, maxTaskDuration=68, avgTaskDuration=68.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788993474_0001_1_00_000000 ], longestDurationTasks=[ task_1452788993474_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:53,750 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788993474_0001_1_00 [EABB36831C1045E1BA44BA3554FB5325] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:53,751 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788993474_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:53,751 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452788993734, Process_Duration=3, Process_End_Time=1452788993737, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 16:29:53,751 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452788993474_0001_1_01 [E515477DC8B447F0B9913E106386FC38] attempt: attempt_1452788993474_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 16:29:53,752 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:53,752 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:53,752 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788993474_0001_00_000001
2016-01-14 16:29:53,752 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E515477DC8B447F0B9913E106386FC38
2016-01-14 16:29:53,753 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788993474_0001_1_01 [E515477DC8B447F0B9913E106386FC38]
2016-01-14 16:29:53,753 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788993474_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788993474_0001_00_000001, stoppedTime=1452788993753, exitStatus=0
2016-01-14 16:29:53,753 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:53,753 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788993474_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:53,754 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788993474_0001_1_00 [EABB36831C1045E1BA44BA3554FB5325] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 16:29:53,754 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788993474_0001_1][Event:TASK_STARTED]: vertexName=E515477DC8B447F0B9913E106386FC38, taskId=task_1452788993474_0001_1_01_000000, scheduledTime=1452788993754, launchTime=1452788993754
2016-01-14 16:29:53,754 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:53,754 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788993474_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:53,755 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452788993474_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 16:29:53,755 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788993474_0001_1_01 [E515477DC8B447F0B9913E106386FC38]
2016-01-14 16:29:53,755 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788993474_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 16:29:53,756 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C821BCDF18DA42D68A88137F9D6F06E9/EDF6AE7F29DD491CA7C7CF7DE020CAF1] count/(1/1) ...latform/testCount3/count3, VertexName: E515477DC8B447F0B9913E106386FC38, VertexParallelism: 1, TaskAttemptID:attempt_1452788993474_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EABB36831C1045E1BA44BA3554FB5325, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A5F3CCDE8D304B74A7A03CC6306B8C29, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:53,756 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788993474_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:53,756 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:53,757 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788993474_0001_00_000002
2016-01-14 16:29:53,757 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788993474_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:53,758 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788993474_0001_00_000002
2016-01-14 16:29:53,758 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 16:29:53,759 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788993474_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:53,759 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788993474_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788993474_0001_00_000002, launchTime=1452788993759
2016-01-14 16:29:53,759 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:53,760 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788993474_0001_00_000002
2016-01-14 16:29:53,761 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788993474_0001_1_01_000000_0] to container: [container_1452788993474_0001_00_000002]
2016-01-14 16:29:53,761 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788993474_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:53,761 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788993474_0001_00_000002 given task: attempt_1452788993474_0001_1_01_000000_0
2016-01-14 16:29:53,761 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788993474_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788993474_0001_1_01_000000_0
2016-01-14 16:29:53,761 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788993474_0001_1_01_000000_0] started. Is using containerId: [container_1452788993474_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 16:29:53,762 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:53,762 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788993474_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E515477DC8B447F0B9913E106386FC38, taskAttemptId=attempt_1452788993474_0001_1_01_000000_0, startTime=1452788993761, containerId=container_1452788993474_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788993474_0001_00_000002/teamcity, completedLogs=
2016-01-14 16:29:53,762 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788993474_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:53,763 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788993474_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:53 Starting to run new task attempt: attempt_1452788993474_0001_1_01_000000_0
2016-01-14 16:29:53,763 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788993474_0001_1_01_000000_0
2016-01-14 16:29:53,763 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452788993474_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 16:29:53,764 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788993474_0001_00_000001: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 16:29:53,763 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:53,764 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452788993474_0001_00_000001, Exiting
2016-01-14 16:29:53,764 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788993474_0001_00_000001
2016-01-14 16:29:53,764 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788993474_0001_00_000001
2016-01-14 16:29:53,764 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:53,766 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:53,769 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:53,769 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:53,769 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788993474_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 16:29:53,770 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452788993474_0001_00_000001 completed successfully
2016-01-14 16:29:53,770 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:53,770 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C821BCDF18DA42D68A88137F9D6F06E9/EDF6AE7F29DD491CA7C7CF7DE020CAF1] count/(1/1) ...latform/testCount3/count3, VertexName: E515477DC8B447F0B9913E106386FC38, VertexParallelism: 1, TaskAttemptID:attempt_1452788993474_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EABB36831C1045E1BA44BA3554FB5325, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A5F3CCDE8D304B74A7A03CC6306B8C29, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:53,770 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:53,771 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788993474_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 16:29:53,771 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788993474_0001_1_01_000000_0
2016-01-14 16:29:53,772 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:53,772 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:53,773 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788993474_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 16:29:53,773 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:53,773 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=EABB36831C1045E1BA44BA3554FB5325, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 16:29:53,774 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:53,774 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: EABB36831C1045E1BA44BA3554FB5325
2016-01-14 16:29:53,775 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A5F3CCDE8D304B74A7A03CC6306B8C29, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 16:29:53,775 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:53,775 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:53,776 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A5F3CCDE8D304B74A7A03CC6306B8C29
2016-01-14 16:29:53,776 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 16:29:53,776 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: A5F3CCDE8D304B74A7A03CC6306B8C29
2016-01-14 16:29:53,776 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 16:29:53,776 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 16:29:53,776 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: EABB36831C1045E1BA44BA3554FB5325
2016-01-14 16:29:53,779 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: EABB36831C1045E1BA44BA3554FB5325
2016-01-14 16:29:53,788 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E515477DC8B447F0B9913E106386FC38, ordinal: 1
2016-01-14 16:29:53,788 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E515477DC8B447F0B9913E106386FC38, mem on start (mb), free: 214, total: 356, max: 672
2016-01-14 16:29:53,788 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:53,788 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:53,788 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:53,792 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 16:29:53,792 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A5F3CCDE8D304B74A7A03CC6306B8C29
2016-01-14 16:29:53,794 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:53,794 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:53,794 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 16:29:53,794 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:53,794 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 16:29:53,795 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 16:29:53,795 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 16:29:53,795 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A5F3CCDE8D304B74A7A03CC6306B8C29, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 16:29:53,795 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, EABB36831C1045E1BA44BA3554FB5325, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 16:29:53,795 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:53,795 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: EABB36831C1045E1BA44BA3554FB5325 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:53,795 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: EABB36831C1045E1BA44BA3554FB5325
2016-01-14 16:29:53,795 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 16:29:53,796 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:53,796 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 16:29:53,796 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 16:29:53,797 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: EABB36831C1045E1BA44BA3554FB5325 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 16:29:53,797 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 16:29:53,798 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 16:29:53,798 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 16:29:53,801 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: EABB36831C1045E1BA44BA3554FB5325
2016-01-14 16:29:53,801 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:53,801 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788993474_0001_1_01_000000_0
2016-01-14 16:29:53,803 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452788993474_0001_1_00_000000_0_10074, runDuration: 56307, ]
2016-01-14 16:29:53,804 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-01-14 16:29:53,805 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452788993474_0001_1_00_000000_0_10074] to DISK_DIRECT, CompressedSize=463, DecompressedSize=459,EndTime=1452788993804, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 16:29:53,805 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : EABB36831C1045E1BA44BA3554FB5325
2016-01-14 16:29:53,805 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: 05E300758FA2457BBCACA86C9BC79B9A
2016-01-14 16:29:53,805 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 16:29:53,805 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3"], id: A5F3CCDE8D304B74A7A03CC6306B8C29
2016-01-14 16:29:53,805 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [EABB36831C1045E1BA44BA3554FB5325] #12 in 1ms
2016-01-14 16:29:53,805 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3"]
2016-01-14 16:29:53,806 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] 05E300758FA2457BBCACA86C9BC79B9A, for 1 inputs
2016-01-14 16:29:53,807 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [EABB36831C1045E1BA44BA3554FB5325] #12, status:false, isInterrupted:false
2016-01-14 16:29:53,807 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 16:29:53,808 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 463 bytes from disk
2016-01-14 16:29:53,808 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 16:29:53,808 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 16:29:53,809 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-01-14 16:29:53,809 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : EABB36831C1045E1BA44BA3554FB5325
2016-01-14 16:29:53,809 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E515477DC8B447F0B9913E106386FC38, all 1 inputs ready in: 00:00:00.003
2016-01-14 16:29:53,810 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@25dccb42; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@42a46d79
2016-01-14 16:29:53,809 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 16:29:53,813 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 16:29:53,813 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 16:29:53,814 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452788993474_0001_1_01_000000_0
2016-01-14 16:29:53,814 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452788993474_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 16:29:53,814 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788993474_0001_1_01_000000_0
2016-01-14 16:29:53,815 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E515477DC8B447F0B9913E106386FC38, mem on close (mb), free: 211, total: 356, max: 672
2016-01-14 16:29:53,815 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: EABB36831C1045E1BA44BA3554FB5325
2016-01-14 16:29:53,815 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 16:29:53,815 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 16:29:53,815 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 16:29:53,815 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 16:29:53,816 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 16:29:53 Completed running task attempt: attempt_1452788993474_0001_1_01_000000_0
2016-01-14 16:29:53,816 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788993474_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:53,817 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452788993809, Process_Duration=5, Process_End_Time=1452788993814, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=17]]
2016-01-14 16:29:53,818 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788993474_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E515477DC8B447F0B9913E106386FC38, taskAttemptId=attempt_1452788993474_0001_1_01_000000_0, startTime=1452788993761, finishTime=1452788993817, timeTaken=56, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788993809, Process_Duration=5, Process_End_Time=1452788993814, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-01-14 16:29:53,818 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:53,818 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:53,819 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788993474_0001_00_000002
2016-01-14 16:29:53,817 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:53,818 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788993474_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:53,819 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788993474_0001_1_01_000000_0
2016-01-14 16:29:53,820 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:53,820 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:53,820 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788993474_0001_1][Event:TASK_FINISHED]: vertexName=E515477DC8B447F0B9913E106386FC38, taskId=task_1452788993474_0001_1_01_000000, startTime=1452788993761, finishTime=1452788993819, timeTaken=58, status=SUCCEEDED, successfulAttemptID=attempt_1452788993474_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788993809, Process_Duration=5, Process_End_Time=1452788993814, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-01-14 16:29:53,820 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788993474_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:53,820 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788993474_0001_1_01 [E515477DC8B447F0B9913E106386FC38] : 1
2016-01-14 16:29:53,821 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788993474_0001_1_01 [E515477DC8B447F0B9913E106386FC38]
2016-01-14 16:29:53,822 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:53,822 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788993474_0001_1][Event:VERTEX_FINISHED]: vertexName=E515477DC8B447F0B9913E106386FC38, vertexId=vertex_1452788993474_0001_1_01, initRequestedTime=1452788993646, initedTime=1452788993656, startRequestedTime=1452788993666, startedTime=1452788993666, finishTime=1452788993821, timeTaken=155, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788993809, Process_Duration=5, Process_End_Time=1452788993814, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1452788993761, firstTasksToStart=[ task_1452788993474_0001_1_01_000000 ], lastTaskFinishTime=1452788993817, lastTasksToFinish=[ task_1452788993474_0001_1_01_000000 ], minTaskDuration=56, maxTaskDuration=56, avgTaskDuration=56.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788993474_0001_1_01_000000 ], longestDurationTasks=[ task_1452788993474_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:53,822 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788993474_0001_1_01 [E515477DC8B447F0B9913E106386FC38] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:53,822 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788993474_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:53,822 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788993474_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788993474_0001_00_000002, stoppedTime=1452788993822, exitStatus=0
2016-01-14 16:29:53,822 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:53,822 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788993474_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:53,823 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788993474_0001_1_01 [E515477DC8B447F0B9913E106386FC38] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 16:29:53,823 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452788993474_0001_1
2016-01-14 16:29:53,824 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788993474_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452788993474_0001_1
2016-01-14 16:29:53,824 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452788993474_0001_1_00 [EABB36831C1045E1BA44BA3554FB5325]
2016-01-14 16:29:53,824 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: A5F3CCDE8D304B74A7A03CC6306B8C29 for vertex: vertex_1452788993474_0001_1_01 [E515477DC8B447F0B9913E106386FC38]
2016-01-14 16:29:53,829 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452788993474_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 16:29:53,829 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788993474_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 16:29:53,829 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452788993474_0001_00_000002, Exiting
2016-01-14 16:29:53,829 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788993474_0001_00_000002
2016-01-14 16:29:53,829 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788993474_0001_00_000002
2016-01-14 16:29:53,830 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788993474_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 16:29:53,830 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452788993474_0001_00_000002 completed successfully
2016-01-14 16:29:53,835 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452788993474_0001_1, queueSize=0
2016-01-14 16:29:53,835 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 16:29:53,836 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788993474_0001_1][Event:DAG_FINISHED]: dagId=dag_1452788993474_0001_1, startTime=1452788993644, finishTime=1452788993833, timeTaken=189, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=940, FILE_BYTES_WRITTEN=777, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=747634688, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905577987543, Process_Duration=8, Process_End_Time=2905577987551, Read_Duration=2, Tuples_Read=40, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-01-14 16:29:53,836 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452788993474_0001_1 finished with state: SUCCEEDED
2016-01-14 16:29:53,836 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788993474_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 16:29:53,836 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452788993474_0001_00_000002
2016-01-14 16:29:53,836 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788993474_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 16:29:53 Completed Dag: dag_1452788993474_0001_1
2016-01-14 16:29:53,837 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452788993474_0001_1, dagState=SUCCEEDED
2016-01-14 16:29:53,837 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788993474_0001_1_post
2016-01-14 16:29:53,837 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:53,838 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 16:29:53,843 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testCount3/count3, applicationId=application_1452788993474_0001
2016-01-14 16:29:53,847 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 16:29:53,847 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 16:29:53,847 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 16:29:53,847 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 16:29:53,847 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 16:29:53,847 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 16:29:53,848 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 16:29:53,848 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 16:29:53,848 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 16:29:53,848 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 16:29:53,849 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 16:29:53,849 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 16:29:53,849 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 16:29:53,855 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/C821BCDF18DA42D68A88137F9D6F06E9/EDF6AE7F29DD491CA7C7CF7DE020CAF1/.tez/application_1452788993474_0001
2016-01-14 16:29:53,856 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] completed in: 00:00.399
2016-01-14 16:29:53,925 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount3/count3
2016-01-14 16:29:53,949 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 16:29:53,949 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 16:29:53,984 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-01-14 16:29:53,992 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-01-14 16:29:53,993 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 16:29:53,993 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 16:29:53,993 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 16:29:54,008 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-01-14 16:29:54,015 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4
2016-01-14 16:29:54,060 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] starting
2016-01-14 16:29:54,060 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 16:29:54,060 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4"]
2016-01-14 16:29:54,060 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] parallel execution of steps is enabled: false
2016-01-14 16:29:54,060 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executing total steps: 1
2016-01-14 16:29:54,061 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] allocating management threads: 1
2016-01-14 16:29:54,061 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] starting step: (1/1) ...latform/testCount4/count4
2016-01-14 16:29:54,063 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 16:29:54,075 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tez session mode enabled: true
2016-01-14 16:29:54,080 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:54,081 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 16:29:54,081 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 16:29:54,092 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/438B6D5880E14FF19CC015A5FDD6F697/BDEF504929BF4B159EE857257D716CD9/.tez/application_1452788994080_0001 doesn't exist and is created
2016-01-14 16:29:54,108 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 16:29:54,108 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 16:29:54,108 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/438B6D5880E14FF19CC015A5FDD6F697/BDEF504929BF4B159EE857257D716CD9/.tez/application_1452788994080_0001
2016-01-14 16:29:54,111 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452788994080_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:54,121 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 16:29:54,121 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 16:29:54,123 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 16:29:54,123 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 16:29:54,123 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/438B6D5880E14FF19CC015A5FDD6F697/BDEF504929BF4B159EE857257D716CD9/.tez/application_1452788994080_0001/localmode-log-dir
2016-01-14 16:29:54,124 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/438B6D5880E14FF19CC015A5FDD6F697/BDEF504929BF4B159EE857257D716CD9/.tez/application_1452788994080_0001/localmode-log-dir/history.txt.appattempt_1452788994080_0001_000000, maxErrors=10
2016-01-14 16:29:54,124 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 16:29:54,124 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452788994080_0001_000000, appSubmitTime=1452788994111, launchTime=1452788994111
2016-01-14 16:29:54,129 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 16:29:54,129 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 16:29:54,130 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 16:29:54,130 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 16:29:54,133 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:34505
2016-01-14 16:29:54,135 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 16:29:54,137 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452788994080_0001_000000, startTime=1452788994137
2016-01-14 16:29:54,137 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 16:29:54,137 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 16:29:54,208 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 16:29:54,208 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 16:29:54,208 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testCount4/count4, applicationId=application_1452788994080_0001, dagName=[438B6D5880E14FF19CC015A5FDD6F697/BDEF504929BF4B159EE857257D716CD9] count/(1/1) ...latform/testCount4/count4
2016-01-14 16:29:54,224 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [438B6D5880E14FF19CC015A5FDD6F697/BDEF504929BF4B159EE857257D716CD9] count/(1/1) ...latform/testCount4/count4
2016-01-14 16:29:54,226 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452788994080_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/438B6D5880E14FF19CC015A5FDD6F697/BDEF504929BF4B159EE857257D716CD9/.tez/application_1452788994080_0001/localmode-log-dir/dag_1452788994080_0001_1.dot
2016-01-14 16:29:54,227 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/438B6D5880E14FF19CC015A5FDD6F697/BDEF504929BF4B159EE857257D716CD9/.tez/application_1452788994080_0001/localmode-log-dir/dag_1452788994080_0001_1-tez-dag.pb.txt
2016-01-14 16:29:54,235 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788994080_0001_1
2016-01-14 16:29:54,236 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:54,236 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [438B6D5880E14FF19CC015A5FDD6F697/BDEF504929BF4B159EE857257D716CD9] count/(1/1) ...latform/testCount4/count4
2016-01-14 16:29:54 Running Dag: dag_1452788994080_0001_1
2016-01-14 16:29:54,249 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994080_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452788994080_0001_1, submitTime=1452788994224
2016-01-14 16:29:54,249 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 16:29:54,250 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D1DCA6C1ECC241F188B0CB878C06ADE2
2016-01-14 16:29:54,251 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 16:29:54,251 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994080_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452788994080_0001_1, initTime=1452788994250
2016-01-14 16:29:54,252 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 16:29:54,253 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788994080_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 16:29:54,253 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994080_0001_1][Event:DAG_STARTED]: dagID=dag_1452788994080_0001_1, startTime=1452788994253
2016-01-14 16:29:54,254 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 16:29:54,254 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788994080_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 16:29:54,254 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D7E8FB9249864E12874B1A2D0DE19799 : {0B5DBD305AF14D5A804A21DFD34E8014={InputName=0B5DBD305AF14D5A804A21DFD34E8014}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 16:29:54,254 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 0B5DBD305AF14D5A804A21DFD34E8014, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 16:29:54,254 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452788994080_0001_1_00 [D7E8FB9249864E12874B1A2D0DE19799]
2016-01-14 16:29:54,253 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 16:29:54,255 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452788994080_0001_1_00 [D7E8FB9249864E12874B1A2D0DE19799]
2016-01-14 16:29:54,255 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452788994080_0001_1_00 [D7E8FB9249864E12874B1A2D0DE19799]
2016-01-14 16:29:54,255 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452788994080_0001_1_00 [D7E8FB9249864E12874B1A2D0DE19799]. Starting root input initializers: 1
2016-01-14 16:29:54,255 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testCount4/count4, applicationId=application_1452788994080_0001, dagName=[438B6D5880E14FF19CC015A5FDD6F697/BDEF504929BF4B159EE857257D716CD9] count/(1/1) ...latform/testCount4/count4
2016-01-14 16:29:54,256 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788994080_0001_1_00 [D7E8FB9249864E12874B1A2D0DE19799] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 16:29:54,256 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452788994080_0001_1_01 [D1DCA6C1ECC241F188B0CB878C06ADE2]
2016-01-14 16:29:54,258 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 0B5DBD305AF14D5A804A21DFD34E8014 on vertex vertex_1452788994080_0001_1_00 [D7E8FB9249864E12874B1A2D0DE19799]
2016-01-14 16:29:54,259 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 16:29:54,260 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452788994080_0001_1_01 [D1DCA6C1ECC241F188B0CB878C06ADE2]
2016-01-14 16:29:54,260 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452788994080_0001_1_01 [D1DCA6C1ECC241F188B0CB878C06ADE2]
2016-01-14 16:29:54,260 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452788994080_0001_1_01 [D1DCA6C1ECC241F188B0CB878C06ADE2]
2016-01-14 16:29:54,260 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=418E15C09E8B42D49CB67E95624DAF61, vertexId=vertex_1452788994080_0001_1_01 [D1DCA6C1ECC241F188B0CB878C06ADE2], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 16:29:54,261 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=418E15C09E8B42D49CB67E95624DAF61, vertexId=vertex_1452788994080_0001_1_01 [D1DCA6C1ECC241F188B0CB878C06ADE2]
2016-01-14 16:29:54,262 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 16:29:54,262 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 0B5DBD305AF14D5A804A21DFD34E8014 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 16:29:54,263 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 16:29:54,263 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=418E15C09E8B42D49CB67E95624DAF61, vertexName=D1DCA6C1ECC241F188B0CB878C06ADE2, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 16:29:54,263 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=418E15C09E8B42D49CB67E95624DAF61, vertexId=vertex_1452788994080_0001_1_01 [D1DCA6C1ECC241F188B0CB878C06ADE2]
2016-01-14 16:29:54,263 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 16:29:54,265 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:54,265 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994080_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D1DCA6C1ECC241F188B0CB878C06ADE2, vertexId=vertex_1452788994080_0001_1_01, initRequestedTime=1452788994256, initedTime=1452788994265, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 16:29:54,265 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788994080_0001_1_01 [D1DCA6C1ECC241F188B0CB878C06ADE2] transitioned from NEW to INITED due to event V_INIT
2016-01-14 16:29:54,266 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-01-14 16:29:54,266 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 0B5DBD305AF14D5A804A21DFD34E8014 on vertex vertex_1452788994080_0001_1_00 [D7E8FB9249864E12874B1A2D0DE19799]
2016-01-14 16:29:54,266 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788994080_0001_1_00 [D7E8FB9249864E12874B1A2D0DE19799] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:54,267 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {0B5DBD305AF14D5A804A21DFD34E8014=forAllWorkUnits=true, update=[1]}
2016-01-14 16:29:54,267 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452788994080_0001_1_00 [D7E8FB9249864E12874B1A2D0DE19799] parallelism set to 1
2016-01-14 16:29:54,267 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788994080_0001_1_00 [D7E8FB9249864E12874B1A2D0DE19799] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:54,268 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994080_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788994080_0001_1_00, eventCount=1
2016-01-14 16:29:54,268 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:54,268 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994080_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D7E8FB9249864E12874B1A2D0DE19799, vertexId=vertex_1452788994080_0001_1_00, initRequestedTime=1452788994254, initedTime=1452788994268, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 16:29:54,269 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788994080_0001_1_00 [D7E8FB9249864E12874B1A2D0DE19799] with distanceFromRoot: 0
2016-01-14 16:29:54,269 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788994080_0001_1_00 [D7E8FB9249864E12874B1A2D0DE19799] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 16:29:54,269 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D7E8FB9249864E12874B1A2D0DE19799
2016-01-14 16:29:54,269 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788994080_0001_1_00 [D7E8FB9249864E12874B1A2D0DE19799]
2016-01-14 16:29:54,270 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:54,270 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994080_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788994080_0001_1_00, startRequestedTime=1452788994269, startedTime=1452788994269
2016-01-14 16:29:54,270 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788994080_0001_1_00 [D7E8FB9249864E12874B1A2D0DE19799] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:54,271 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:54,271 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994080_0001_1][Event:TASK_STARTED]: vertexName=D7E8FB9249864E12874B1A2D0DE19799, taskId=task_1452788994080_0001_1_00_000000, scheduledTime=1452788994271, launchTime=1452788994271
2016-01-14 16:29:54,271 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788994080_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:54,271 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452788994080_0001_1_00 for vertex: vertex_1452788994080_0001_1_01 [D1DCA6C1ECC241F188B0CB878C06ADE2] numStartedSources: 1 numSources: 1
2016-01-14 16:29:54,272 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788994080_0001_1_01 [D1DCA6C1ECC241F188B0CB878C06ADE2] with distanceFromRoot: 1
2016-01-14 16:29:54,272 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788994080_0001_1_00 [D7E8FB9249864E12874B1A2D0DE19799]
2016-01-14 16:29:54,272 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D7E8FB9249864E12874B1A2D0DE19799 in vertex: D1DCA6C1ECC241F188B0CB878C06ADE2
2016-01-14 16:29:54,272 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: D1DCA6C1ECC241F188B0CB878C06ADE2 with 1 source tasks and 1 pending tasks
2016-01-14 16:29:54,273 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:54,273 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994080_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788994080_0001_1_01, startRequestedTime=1452788994272, startedTime=1452788994272
2016-01-14 16:29:54,273 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] submitted tez dag to app master: application_1452788994080_0001, with dag id: dag_1452788994080_0001_1
2016-01-14 16:29:54,273 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788994080_0001_1_01 [D1DCA6C1ECC241F188B0CB878C06ADE2] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:54,274 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [438B6D5880E14FF19CC015A5FDD6F697/BDEF504929BF4B159EE857257D716CD9] count/(1/1) ...latform/testCount4/count4, VertexName: D7E8FB9249864E12874B1A2D0DE19799, VertexParallelism: 1, TaskAttemptID:attempt_1452788994080_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0B5DBD305AF14D5A804A21DFD34E8014, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D1DCA6C1ECC241F188B0CB878C06ADE2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 16:29:54,274 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788994080_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:54,275 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:54,275 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 16:29:54,276 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 16:29:54,276 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 16:29:54,277 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788994080_0001_00_000001
2016-01-14 16:29:54,277 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788994080_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:54,279 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788994080_0001_00_000001
2016-01-14 16:29:54,279 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 16:29:54,280 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788994080_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:54,280 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:54,281 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994080_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788994080_0001_00_000001, launchTime=1452788994279
2016-01-14 16:29:54,281 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788994080_0001_00_000001
2016-01-14 16:29:54,282 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788994080_0001_1_00_000000_0] to container: [container_1452788994080_0001_00_000001]
2016-01-14 16:29:54,282 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788994080_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:54,282 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788994080_0001_00_000001 given task: attempt_1452788994080_0001_1_00_000000_0
2016-01-14 16:29:54,283 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788994080_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788994080_0001_1_00_000000_0
2016-01-14 16:29:54,282 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788994080_0001_1_00_000000_0] started. Is using containerId: [container_1452788994080_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 16:29:54,283 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:54 Starting to run new task attempt: attempt_1452788994080_0001_1_00_000000_0
2016-01-14 16:29:54,283 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788994080_0001_1_00_000000_0
2016-01-14 16:29:54,283 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:54,284 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:54,284 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:54,284 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:54,284 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:54,284 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:54,284 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [438B6D5880E14FF19CC015A5FDD6F697/BDEF504929BF4B159EE857257D716CD9] count/(1/1) ...latform/testCount4/count4, VertexName: D7E8FB9249864E12874B1A2D0DE19799, VertexParallelism: 1, TaskAttemptID:attempt_1452788994080_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0B5DBD305AF14D5A804A21DFD34E8014, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D1DCA6C1ECC241F188B0CB878C06ADE2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 16:29:54,284 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994080_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D7E8FB9249864E12874B1A2D0DE19799, taskAttemptId=attempt_1452788994080_0001_1_00_000000_0, startTime=1452788994282, containerId=container_1452788994080_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788994080_0001_00_000001/teamcity, completedLogs=
2016-01-14 16:29:54,285 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:54,285 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788994080_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:54,285 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788994080_0001_1_00_000000_0
2016-01-14 16:29:54,286 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788994080_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 16:29:54,286 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:54,286 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:54,286 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788994080_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:54,287 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:54,287 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D1DCA6C1ECC241F188B0CB878C06ADE2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 16:29:54,287 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=0B5DBD305AF14D5A804A21DFD34E8014, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 16:29:54,288 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:54,288 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:54,288 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D1DCA6C1ECC241F188B0CB878C06ADE2
2016-01-14 16:29:54,289 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 0B5DBD305AF14D5A804A21DFD34E8014
2016-01-14 16:29:54,289 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 0B5DBD305AF14D5A804A21DFD34E8014
2016-01-14 16:29:54,291 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:54,291 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 16:29:54,291 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 16:29:54,291 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 0B5DBD305AF14D5A804A21DFD34E8014
2016-01-14 16:29:54,292 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: D1DCA6C1ECC241F188B0CB878C06ADE2
2016-01-14 16:29:54,292 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 0B5DBD305AF14D5A804A21DFD34E8014
2016-01-14 16:29:54,294 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D1DCA6C1ECC241F188B0CB878C06ADE2
2016-01-14 16:29:54,300 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D7E8FB9249864E12874B1A2D0DE19799, ordinal: 0
2016-01-14 16:29:54,300 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D7E8FB9249864E12874B1A2D0DE19799, mem on start (mb), free: 122, total: 356, max: 672
2016-01-14 16:29:54,300 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:54,300 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:54,301 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:54,301 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:54,301 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:54,301 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 16:29:54,301 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:54,301 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 16:29:54,301 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 16:29:54,301 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 16:29:54,301 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 0B5DBD305AF14D5A804A21DFD34E8014, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 16:29:54,301 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D1DCA6C1ECC241F188B0CB878C06ADE2, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 16:29:54,302 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:54,302 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 0B5DBD305AF14D5A804A21DFD34E8014 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:54,302 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 16:29:54,303 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:54,303 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 0B5DBD305AF14D5A804A21DFD34E8014
2016-01-14 16:29:54,303 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:54,303 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 0B5DBD305AF14D5A804A21DFD34E8014
2016-01-14 16:29:54,303 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:54,303 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788994080_0001_1_00_000000_0
2016-01-14 16:29:54,304 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 16:29:54,304 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6dc33d1d
2016-01-14 16:29:54,305 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6dc33d1d
2016-01-14 16:29:54,306 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 16:29:54,306 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 16:29:54,307 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 16:29:54,308 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: 0B5DBD305AF14D5A804A21DFD34E8014
2016-01-14 16:29:54,308 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: F9C9BBBE340E44B2A697EBDDDA1EBECA
2016-01-14 16:29:54,308 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] F9C9BBBE340E44B2A697EBDDDA1EBECA
2016-01-14 16:29:54,309 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@fcb9f06; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@1354b04; comparator=cascading.tuple.hadoop.util.TupleComparator@51cff43a; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 16:29:54,310 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 16:29:54,310 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 16:29:54,310 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 16:29:54,335 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 16:29:54,335 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 16:29:54,335 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 16:29:54,335 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 16:29:54,335 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 16:29:54,336 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 16:29:54,336 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:54,337 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D7E8FB9249864E12874B1A2D0DE19799, all 1 inputs ready in: 00:00:00.000
2016-01-14 16:29:54,339 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788994080_0001_1_00_000000_0
2016-01-14 16:29:54,339 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D7E8FB9249864E12874B1A2D0DE19799, mem on close (mb), free: 218, total: 357, max: 672
2016-01-14 16:29:54,339 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 16:29:54,339 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 16:29:54,340 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2016-01-14 16:29:54,340 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 16:29:54,343 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 16:29:54 Completed running task attempt: attempt_1452788994080_0001_1_00_000000_0
2016-01-14 16:29:54,346 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788994080_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:54,346 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994080_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788994080_0001_1_00, eventCount=1
2016-01-14 16:29:54,346 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374341632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452788994337, Process_Duration=2, Process_End_Time=1452788994339, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 16:29:54,347 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:54,347 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:54,347 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788994080_0001_00_000001
2016-01-14 16:29:54,347 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:54,347 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994080_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D7E8FB9249864E12874B1A2D0DE19799, taskAttemptId=attempt_1452788994080_0001_1_00_000000_0, startTime=1452788994282, finishTime=1452788994347, timeTaken=65, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374341632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788994337, Process_Duration=2, Process_End_Time=1452788994339, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 16:29:54,347 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788994080_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:54,348 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788994080_0001_1_00_000000_0
2016-01-14 16:29:54,348 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:54,348 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994080_0001_1][Event:TASK_FINISHED]: vertexName=D7E8FB9249864E12874B1A2D0DE19799, taskId=task_1452788994080_0001_1_00_000000, startTime=1452788994282, finishTime=1452788994348, timeTaken=66, status=SUCCEEDED, successfulAttemptID=attempt_1452788994080_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374341632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788994337, Process_Duration=2, Process_End_Time=1452788994339, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 16:29:54,348 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788994080_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:54,349 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788994080_0001_1_00 [D7E8FB9249864E12874B1A2D0DE19799] : 1
2016-01-14 16:29:54,349 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788994080_0001_1_00 [D7E8FB9249864E12874B1A2D0DE19799]
2016-01-14 16:29:54,349 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:54,350 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:54,350 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994080_0001_1][Event:VERTEX_FINISHED]: vertexName=D7E8FB9249864E12874B1A2D0DE19799, vertexId=vertex_1452788994080_0001_1_00, initRequestedTime=1452788994254, initedTime=1452788994268, startRequestedTime=1452788994269, startedTime=1452788994269, finishTime=1452788994349, timeTaken=80, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374341632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788994337, Process_Duration=2, Process_End_Time=1452788994339, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452788994282, firstTasksToStart=[ task_1452788994080_0001_1_00_000000 ], lastTaskFinishTime=1452788994347, lastTasksToFinish=[ task_1452788994080_0001_1_00_000000 ], minTaskDuration=65, maxTaskDuration=65, avgTaskDuration=65.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788994080_0001_1_00_000000 ], longestDurationTasks=[ task_1452788994080_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:54,350 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788994080_0001_1_00 [D7E8FB9249864E12874B1A2D0DE19799] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:54,350 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452788994080_0001_1_01 [D1DCA6C1ECC241F188B0CB878C06ADE2] attempt: attempt_1452788994080_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 16:29:54,351 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D1DCA6C1ECC241F188B0CB878C06ADE2
2016-01-14 16:29:54,351 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788994080_0001_1_01 [D1DCA6C1ECC241F188B0CB878C06ADE2]
2016-01-14 16:29:54,351 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788994080_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:54,351 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788994080_0001_1_00 [D7E8FB9249864E12874B1A2D0DE19799] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 16:29:54,355 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994080_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788994080_0001_00_000001, stoppedTime=1452788994355, exitStatus=0
2016-01-14 16:29:54,355 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788994080_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:54,355 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:54,355 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994080_0001_1][Event:TASK_STARTED]: vertexName=D1DCA6C1ECC241F188B0CB878C06ADE2, taskId=task_1452788994080_0001_1_01_000000, scheduledTime=1452788994355, launchTime=1452788994355
2016-01-14 16:29:54,356 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:54,356 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788994080_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:54,356 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452788994080_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 16:29:54,356 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788994080_0001_1_01 [D1DCA6C1ECC241F188B0CB878C06ADE2]
2016-01-14 16:29:54,357 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788994080_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 16:29:54,357 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [438B6D5880E14FF19CC015A5FDD6F697/BDEF504929BF4B159EE857257D716CD9] count/(1/1) ...latform/testCount4/count4, VertexName: D1DCA6C1ECC241F188B0CB878C06ADE2, VertexParallelism: 1, TaskAttemptID:attempt_1452788994080_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D7E8FB9249864E12874B1A2D0DE19799, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=418E15C09E8B42D49CB67E95624DAF61, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:54,357 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788994080_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:54,357 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:54,358 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788994080_0001_00_000002
2016-01-14 16:29:54,358 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788994080_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:54,358 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452788994080_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 16:29:54,358 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788994080_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 16:29:54,358 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452788994080_0001_00_000001, Exiting
2016-01-14 16:29:54,358 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788994080_0001_00_000001
2016-01-14 16:29:54,358 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788994080_0001_00_000001
2016-01-14 16:29:54,359 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788994080_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 16:29:54,359 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788994080_0001_00_000002
2016-01-14 16:29:54,361 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 16:29:54,360 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452788994080_0001_00_000001 completed successfully
2016-01-14 16:29:54,361 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788994080_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 16:29:54,363 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788994080_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:54,363 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:54,363 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994080_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788994080_0001_00_000002, launchTime=1452788994363
2016-01-14 16:29:54,364 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788994080_0001_00_000002
2016-01-14 16:29:54,364 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788994080_0001_1_01_000000_0] to container: [container_1452788994080_0001_00_000002]
2016-01-14 16:29:54,364 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788994080_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:54,365 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788994080_0001_00_000002 given task: attempt_1452788994080_0001_1_01_000000_0
2016-01-14 16:29:54,365 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788994080_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788994080_0001_1_01_000000_0
2016-01-14 16:29:54,365 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788994080_0001_1_01_000000_0] started. Is using containerId: [container_1452788994080_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 16:29:54,365 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994080_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D1DCA6C1ECC241F188B0CB878C06ADE2, taskAttemptId=attempt_1452788994080_0001_1_01_000000_0, startTime=1452788994365, containerId=container_1452788994080_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788994080_0001_00_000002/teamcity, completedLogs=
2016-01-14 16:29:54,365 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788994080_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:54,366 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788994080_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:54,366 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:54 Starting to run new task attempt: attempt_1452788994080_0001_1_01_000000_0
2016-01-14 16:29:54,366 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788994080_0001_1_01_000000_0
2016-01-14 16:29:54,366 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:54,366 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:54,367 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:54,367 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:54,367 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:54,367 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:54,367 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [438B6D5880E14FF19CC015A5FDD6F697/BDEF504929BF4B159EE857257D716CD9] count/(1/1) ...latform/testCount4/count4, VertexName: D1DCA6C1ECC241F188B0CB878C06ADE2, VertexParallelism: 1, TaskAttemptID:attempt_1452788994080_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D7E8FB9249864E12874B1A2D0DE19799, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=418E15C09E8B42D49CB67E95624DAF61, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:54,368 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:54,369 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788994080_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 16:29:54,369 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788994080_0001_1_01_000000_0
2016-01-14 16:29:54,369 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:54,372 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:54,372 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:54,372 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D7E8FB9249864E12874B1A2D0DE19799, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 16:29:54,373 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:54,373 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D7E8FB9249864E12874B1A2D0DE19799
2016-01-14 16:29:54,375 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=418E15C09E8B42D49CB67E95624DAF61, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 16:29:54,375 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 16:29:54,376 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 16:29:54,376 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D7E8FB9249864E12874B1A2D0DE19799
2016-01-14 16:29:54,376 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D7E8FB9249864E12874B1A2D0DE19799
2016-01-14 16:29:54,375 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:54,375 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:54,376 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 418E15C09E8B42D49CB67E95624DAF61
2016-01-14 16:29:54,377 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 16:29:54,377 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 418E15C09E8B42D49CB67E95624DAF61
2016-01-14 16:29:54,386 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D1DCA6C1ECC241F188B0CB878C06ADE2, ordinal: 1
2016-01-14 16:29:54,386 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D1DCA6C1ECC241F188B0CB878C06ADE2, mem on start (mb), free: 212, total: 357, max: 672
2016-01-14 16:29:54,386 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:54,386 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:54,386 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:54,391 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 16:29:54,391 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 418E15C09E8B42D49CB67E95624DAF61
2016-01-14 16:29:54,392 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:54,392 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:54,393 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 16:29:54,393 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 16:29:54,393 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:54,394 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 16:29:54,394 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 16:29:54,394 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D7E8FB9249864E12874B1A2D0DE19799, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 16:29:54,395 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 418E15C09E8B42D49CB67E95624DAF61, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 16:29:54,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:54,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D7E8FB9249864E12874B1A2D0DE19799
2016-01-14 16:29:54,396 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 16:29:54,396 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 16:29:54,396 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D7E8FB9249864E12874B1A2D0DE19799 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 16:29:54,396 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 16:29:54,396 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 16:29:54,396 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D7E8FB9249864E12874B1A2D0DE19799 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:54,397 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 16:29:54,397 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 16:29:54,397 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:54,398 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D7E8FB9249864E12874B1A2D0DE19799
2016-01-14 16:29:54,398 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:54,399 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788994080_0001_1_01_000000_0
2016-01-14 16:29:54,401 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452788994080_0001_1_00_000000_0_10080, runDuration: 56022, ]
2016-01-14 16:29:54,401 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-01-14 16:29:54,402 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: F9C9BBBE340E44B2A697EBDDDA1EBECA
2016-01-14 16:29:54,402 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4"], id: 418E15C09E8B42D49CB67E95624DAF61
2016-01-14 16:29:54,402 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452788994080_0001_1_00_000000_0_10080] to DISK_DIRECT, CompressedSize=463, DecompressedSize=459,EndTime=1452788994402, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 16:29:54,403 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D7E8FB9249864E12874B1A2D0DE19799
2016-01-14 16:29:54,403 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4"]
2016-01-14 16:29:54,403 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 16:29:54,403 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] F9C9BBBE340E44B2A697EBDDDA1EBECA, for 1 inputs
2016-01-14 16:29:54,403 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [D7E8FB9249864E12874B1A2D0DE19799] #13 in 1ms
2016-01-14 16:29:54,403 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D7E8FB9249864E12874B1A2D0DE19799] #13, status:false, isInterrupted:false
2016-01-14 16:29:54,404 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 16:29:54,404 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 463 bytes from disk
2016-01-14 16:29:54,404 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 16:29:54,404 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 16:29:54,405 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-01-14 16:29:54,405 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D7E8FB9249864E12874B1A2D0DE19799
2016-01-14 16:29:54,405 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 16:29:54,405 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D1DCA6C1ECC241F188B0CB878C06ADE2, all 1 inputs ready in: 00:00:00.002
2016-01-14 16:29:54,406 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@453a7cca; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@2d6d548
2016-01-14 16:29:54,411 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 16:29:54,411 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 16:29:54,411 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452788994080_0001_1_01_000000_0
2016-01-14 16:29:54,412 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452788994080_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 16:29:54,412 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788994080_0001_1_01_000000_0
2016-01-14 16:29:54,412 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D1DCA6C1ECC241F188B0CB878C06ADE2, mem on close (mb), free: 209, total: 357, max: 672
2016-01-14 16:29:54,413 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D7E8FB9249864E12874B1A2D0DE19799
2016-01-14 16:29:54,413 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 16:29:54,413 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 16:29:54,414 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 16:29:54,414 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 16:29:54,414 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 16:29:54 Completed running task attempt: attempt_1452788994080_0001_1_01_000000_0
2016-01-14 16:29:54,415 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788994080_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:54,416 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374341632, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452788994406, Process_Duration=6, Process_End_Time=1452788994412, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=17]]
2016-01-14 16:29:54,416 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:54,416 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994080_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D1DCA6C1ECC241F188B0CB878C06ADE2, taskAttemptId=attempt_1452788994080_0001_1_01_000000_0, startTime=1452788994365, finishTime=1452788994416, timeTaken=51, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374341632, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788994406, Process_Duration=6, Process_End_Time=1452788994412, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-01-14 16:29:54,417 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788994080_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:54,417 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788994080_0001_1_01_000000_0
2016-01-14 16:29:54,417 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:54,418 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:54,418 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994080_0001_1][Event:TASK_FINISHED]: vertexName=D1DCA6C1ECC241F188B0CB878C06ADE2, taskId=task_1452788994080_0001_1_01_000000, startTime=1452788994365, finishTime=1452788994417, timeTaken=52, status=SUCCEEDED, successfulAttemptID=attempt_1452788994080_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374341632, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788994406, Process_Duration=6, Process_End_Time=1452788994412, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-01-14 16:29:54,418 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788994080_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:54,418 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788994080_0001_1_01 [D1DCA6C1ECC241F188B0CB878C06ADE2] : 1
2016-01-14 16:29:54,418 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788994080_0001_1_01 [D1DCA6C1ECC241F188B0CB878C06ADE2]
2016-01-14 16:29:54,418 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:54,418 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788994080_0001_00_000002
2016-01-14 16:29:54,419 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:54,419 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994080_0001_1][Event:VERTEX_FINISHED]: vertexName=D1DCA6C1ECC241F188B0CB878C06ADE2, vertexId=vertex_1452788994080_0001_1_01, initRequestedTime=1452788994256, initedTime=1452788994265, startRequestedTime=1452788994272, startedTime=1452788994272, finishTime=1452788994418, timeTaken=146, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374341632, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788994406, Process_Duration=6, Process_End_Time=1452788994412, Read_Duration=2, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1452788994365, firstTasksToStart=[ task_1452788994080_0001_1_01_000000 ], lastTaskFinishTime=1452788994416, lastTasksToFinish=[ task_1452788994080_0001_1_01_000000 ], minTaskDuration=51, maxTaskDuration=51, avgTaskDuration=51.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788994080_0001_1_01_000000 ], longestDurationTasks=[ task_1452788994080_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:54,420 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788994080_0001_1_01 [D1DCA6C1ECC241F188B0CB878C06ADE2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:54,420 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:54,421 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788994080_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:54,421 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994080_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788994080_0001_00_000002, stoppedTime=1452788994421, exitStatus=0
2016-01-14 16:29:54,421 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788994080_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:54,421 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788994080_0001_1_01 [D1DCA6C1ECC241F188B0CB878C06ADE2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 16:29:54,421 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452788994080_0001_1
2016-01-14 16:29:54,422 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994080_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452788994080_0001_1
2016-01-14 16:29:54,422 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 418E15C09E8B42D49CB67E95624DAF61 for vertex: vertex_1452788994080_0001_1_01 [D1DCA6C1ECC241F188B0CB878C06ADE2]
2016-01-14 16:29:54,422 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:54,428 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452788994080_0001_1_00 [D7E8FB9249864E12874B1A2D0DE19799]
2016-01-14 16:29:54,429 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452788994080_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 16:29:54,429 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788994080_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 16:29:54,429 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452788994080_0001_00_000002, Exiting
2016-01-14 16:29:54,429 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788994080_0001_00_000002
2016-01-14 16:29:54,429 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788994080_0001_00_000002
2016-01-14 16:29:54,430 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788994080_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 16:29:54,430 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452788994080_0001_00_000002 completed successfully
2016-01-14 16:29:54,430 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452788994080_0001_1, queueSize=0
2016-01-14 16:29:54,430 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 16:29:54,431 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994080_0001_1][Event:DAG_FINISHED]: dagId=dag_1452788994080_0001_1, startTime=1452788994253, finishTime=1452788994428, timeTaken=175, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=940, FILE_BYTES_WRITTEN=777, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=748683264, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905577988743, Process_Duration=8, Process_End_Time=2905577988751, Read_Duration=2, Tuples_Read=40, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-01-14 16:29:54,431 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452788994080_0001_1 finished with state: SUCCEEDED
2016-01-14 16:29:54,432 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788994080_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 16:29:54,433 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452788994080_0001_00_000002
2016-01-14 16:29:54,433 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788994080_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 16:29:54 Completed Dag: dag_1452788994080_0001_1
2016-01-14 16:29:54,434 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452788994080_0001_1, dagState=SUCCEEDED
2016-01-14 16:29:54,434 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788994080_0001_1_post
2016-01-14 16:29:54,434 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:54,435 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 16:29:54,439 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testCount4/count4, applicationId=application_1452788994080_0001
2016-01-14 16:29:54,441 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 16:29:54,441 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 16:29:54,441 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 16:29:54,441 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 16:29:54,441 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 16:29:54,441 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 16:29:54,442 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 16:29:54,442 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 16:29:54,442 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 16:29:54,442 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 16:29:54,442 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 16:29:54,443 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 16:29:54,443 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 16:29:54,448 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/438B6D5880E14FF19CC015A5FDD6F697/BDEF504929BF4B159EE857257D716CD9/.tez/application_1452788994080_0001
2016-01-14 16:29:54,448 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] completed in: 00:00.388
2016-01-14 16:29:54,449 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 16:29:54,449 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 16:29:54,513 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount4/count4
2016-01-14 16:29:54,535 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 16:29:54,536 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 16:29:54,578 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-01-14 16:29:54,578 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-01-14 16:29:54,579 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 16:29:54,579 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 16:29:54,579 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 16:29:54,594 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-01-14 16:29:54,594 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter
2016-01-14 16:29:54,602 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 16:29:54,603 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 16:29:54,603 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter"]
2016-01-14 16:29:54,603 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 16:29:54,604 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 16:29:54,604 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 16:29:54,604 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...latform/testFilter/filter
2016-01-14 16:29:54,604 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 16:29:54,616 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 16:29:54,620 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:54,621 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 16:29:54,621 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 16:29:54,630 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0687247E815F4851A88137658662BFF1/F2BECC5A70A4482A8AF418028EAA4065/.tez/application_1452788994620_0001 doesn't exist and is created
2016-01-14 16:29:54,646 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 16:29:54,646 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 16:29:54,647 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0687247E815F4851A88137658662BFF1/F2BECC5A70A4482A8AF418028EAA4065/.tez/application_1452788994620_0001
2016-01-14 16:29:54,650 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452788994620_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:54,660 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 16:29:54,660 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 16:29:54,662 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 16:29:54,662 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 16:29:54,662 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0687247E815F4851A88137658662BFF1/F2BECC5A70A4482A8AF418028EAA4065/.tez/application_1452788994620_0001/localmode-log-dir
2016-01-14 16:29:54,663 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0687247E815F4851A88137658662BFF1/F2BECC5A70A4482A8AF418028EAA4065/.tez/application_1452788994620_0001/localmode-log-dir/history.txt.appattempt_1452788994620_0001_000000, maxErrors=10
2016-01-14 16:29:54,663 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 16:29:54,664 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452788994620_0001_000000, appSubmitTime=1452788994649, launchTime=1452788994649
2016-01-14 16:29:54,666 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 16:29:54,667 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 16:29:54,668 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 16:29:54,668 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 16:29:54,671 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:51389
2016-01-14 16:29:54,672 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 16:29:54,674 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452788994620_0001_000000, startTime=1452788994674
2016-01-14 16:29:54,674 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 16:29:54,674 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 16:29:54,746 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 16:29:54,747 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 16:29:54,747 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testFilter/filter, applicationId=application_1452788994620_0001, dagName=[0687247E815F4851A88137658662BFF1/F2BECC5A70A4482A8AF418028EAA4065] test/(1/1) ...latform/testFilter/filter
2016-01-14 16:29:54,762 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [0687247E815F4851A88137658662BFF1/F2BECC5A70A4482A8AF418028EAA4065] test/(1/1) ...latform/testFilter/filter
2016-01-14 16:29:54,764 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452788994620_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0687247E815F4851A88137658662BFF1/F2BECC5A70A4482A8AF418028EAA4065/.tez/application_1452788994620_0001/localmode-log-dir/dag_1452788994620_0001_1.dot
2016-01-14 16:29:54,764 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0687247E815F4851A88137658662BFF1/F2BECC5A70A4482A8AF418028EAA4065/.tez/application_1452788994620_0001/localmode-log-dir/dag_1452788994620_0001_1-tez-dag.pb.txt
2016-01-14 16:29:54,770 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788994620_0001_1
2016-01-14 16:29:54,770 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:54,771 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [0687247E815F4851A88137658662BFF1/F2BECC5A70A4482A8AF418028EAA4065] test/(1/1) ...latform/testFilter/filter
2016-01-14 16:29:54 Running Dag: dag_1452788994620_0001_1
2016-01-14 16:29:54,783 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 16:29:54,783 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994620_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452788994620_0001_1, submitTime=1452788994762
2016-01-14 16:29:54,784 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex EBC5870300144420851B41AA31EC272E
2016-01-14 16:29:54,784 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 16:29:54,784 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 16:29:54,785 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994620_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452788994620_0001_1, initTime=1452788994784
2016-01-14 16:29:54,785 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788994620_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 16:29:54,786 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994620_0001_1][Event:DAG_STARTED]: dagID=dag_1452788994620_0001_1, startTime=1452788994785
2016-01-14 16:29:54,786 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 16:29:54,786 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 16:29:54,786 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788994620_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 16:29:54,786 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testFilter/filter, applicationId=application_1452788994620_0001, dagName=[0687247E815F4851A88137658662BFF1/F2BECC5A70A4482A8AF418028EAA4065] test/(1/1) ...latform/testFilter/filter
2016-01-14 16:29:54,787 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: EBC5870300144420851B41AA31EC272E : {4001295E0A6D47EF95A734E1194DB0E4={InputName=4001295E0A6D47EF95A734E1194DB0E4}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 16:29:54,788 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 4001295E0A6D47EF95A734E1194DB0E4, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 16:29:54,788 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452788994620_0001_1_00 [EBC5870300144420851B41AA31EC272E]
2016-01-14 16:29:54,788 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452788994620_0001_1_00 [EBC5870300144420851B41AA31EC272E]
2016-01-14 16:29:54,788 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452788994620_0001_1_00 [EBC5870300144420851B41AA31EC272E]
2016-01-14 16:29:54,788 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452788994620_0001_1_00 [EBC5870300144420851B41AA31EC272E]. Starting root input initializers: 1
2016-01-14 16:29:54,792 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788994620_0001_1_00 [EBC5870300144420851B41AA31EC272E] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 16:29:54,792 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 4001295E0A6D47EF95A734E1194DB0E4 on vertex vertex_1452788994620_0001_1_00 [EBC5870300144420851B41AA31EC272E]
2016-01-14 16:29:54,794 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 16:29:54,794 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 4001295E0A6D47EF95A734E1194DB0E4 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 16:29:54,794 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 16:29:54,795 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 16:29:54,796 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-01-14 16:29:54,797 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 4001295E0A6D47EF95A734E1194DB0E4 on vertex vertex_1452788994620_0001_1_00 [EBC5870300144420851B41AA31EC272E]
2016-01-14 16:29:54,797 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788994620_0001_1_00 [EBC5870300144420851B41AA31EC272E] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:54,797 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {4001295E0A6D47EF95A734E1194DB0E4=forAllWorkUnits=true, update=[1]}
2016-01-14 16:29:54,797 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452788994620_0001_1_00 [EBC5870300144420851B41AA31EC272E] parallelism set to 1
2016-01-14 16:29:54,797 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788994620_0001_1_00 [EBC5870300144420851B41AA31EC272E] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:54,798 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994620_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788994620_0001_1_00, eventCount=1
2016-01-14 16:29:54,798 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452788994620_0001_1_00 [EBC5870300144420851B41AA31EC272E]
2016-01-14 16:29:54,798 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=ED8345720D7C424EB8903558ABB8EF6F, vertexId=vertex_1452788994620_0001_1_00 [EBC5870300144420851B41AA31EC272E], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 16:29:54,798 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=ED8345720D7C424EB8903558ABB8EF6F, vertexId=vertex_1452788994620_0001_1_00 [EBC5870300144420851B41AA31EC272E]
2016-01-14 16:29:54,799 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=ED8345720D7C424EB8903558ABB8EF6F, vertexName=EBC5870300144420851B41AA31EC272E, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 16:29:54,800 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=ED8345720D7C424EB8903558ABB8EF6F, vertexId=vertex_1452788994620_0001_1_00 [EBC5870300144420851B41AA31EC272E]
2016-01-14 16:29:54,801 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994620_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EBC5870300144420851B41AA31EC272E, vertexId=vertex_1452788994620_0001_1_00, initRequestedTime=1452788994787, initedTime=1452788994801, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 16:29:54,801 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788994620_0001_1_00 [EBC5870300144420851B41AA31EC272E] with distanceFromRoot: 0
2016-01-14 16:29:54,801 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:54,802 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788994620_0001_1_00 [EBC5870300144420851B41AA31EC272E] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 16:29:54,802 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in EBC5870300144420851B41AA31EC272E
2016-01-14 16:29:54,802 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788994620_0001_1_00 [EBC5870300144420851B41AA31EC272E]
2016-01-14 16:29:54,803 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994620_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788994620_0001_1_00, startRequestedTime=1452788994802, startedTime=1452788994802
2016-01-14 16:29:54,803 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:54,803 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788994620_0001_1_00 [EBC5870300144420851B41AA31EC272E] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:54,804 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:54,804 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994620_0001_1][Event:TASK_STARTED]: vertexName=EBC5870300144420851B41AA31EC272E, taskId=task_1452788994620_0001_1_00_000000, scheduledTime=1452788994804, launchTime=1452788994804
2016-01-14 16:29:54,804 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788994620_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:54,805 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788994620_0001_1_00 [EBC5870300144420851B41AA31EC272E]
2016-01-14 16:29:54,805 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [0687247E815F4851A88137658662BFF1/F2BECC5A70A4482A8AF418028EAA4065] test/(1/1) ...latform/testFilter/filter, VertexName: EBC5870300144420851B41AA31EC272E, VertexParallelism: 1, TaskAttemptID:attempt_1452788994620_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4001295E0A6D47EF95A734E1194DB0E4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=ED8345720D7C424EB8903558ABB8EF6F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:54,805 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788994620_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:54,806 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:54,806 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 16:29:54,807 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 16:29:54,807 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 16:29:54,807 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788994620_0001_00_000001
2016-01-14 16:29:54,808 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788994620_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:54,808 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452788994620_0001, with dag id: dag_1452788994620_0001_1
2016-01-14 16:29:54,811 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788994620_0001_00_000001
2016-01-14 16:29:54,811 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 16:29:54,812 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788994620_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:54,812 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:54,813 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994620_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788994620_0001_00_000001, launchTime=1452788994812
2016-01-14 16:29:54,814 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788994620_0001_00_000001
2016-01-14 16:29:54,814 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788994620_0001_1_00_000000_0] to container: [container_1452788994620_0001_00_000001]
2016-01-14 16:29:54,815 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788994620_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:54,816 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788994620_0001_1_00_000000_0] started. Is using containerId: [container_1452788994620_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 16:29:54,816 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994620_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EBC5870300144420851B41AA31EC272E, taskAttemptId=attempt_1452788994620_0001_1_00_000000_0, startTime=1452788994815, containerId=container_1452788994620_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788994620_0001_00_000001/teamcity, completedLogs=
2016-01-14 16:29:54,816 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788994620_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:54,816 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788994620_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:54,817 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:54,818 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788994620_0001_00_000001 given task: attempt_1452788994620_0001_1_00_000000_0
2016-01-14 16:29:54,818 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788994620_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788994620_0001_1_00_000000_0
2016-01-14 16:29:54 Starting to run new task attempt: attempt_1452788994620_0001_1_00_000000_0
2016-01-14 16:29:54,819 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788994620_0001_1_00_000000_0
2016-01-14 16:29:54,819 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:54,819 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:54,819 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:54,819 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:54,819 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:54,819 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:54,820 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [0687247E815F4851A88137658662BFF1/F2BECC5A70A4482A8AF418028EAA4065] test/(1/1) ...latform/testFilter/filter, VertexName: EBC5870300144420851B41AA31EC272E, VertexParallelism: 1, TaskAttemptID:attempt_1452788994620_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4001295E0A6D47EF95A734E1194DB0E4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=ED8345720D7C424EB8903558ABB8EF6F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:54,820 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:54,821 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788994620_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 16:29:54,821 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788994620_0001_1_00_000000_0
2016-01-14 16:29:54,821 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:54,821 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:54,822 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=4001295E0A6D47EF95A734E1194DB0E4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 16:29:54,822 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=ED8345720D7C424EB8903558ABB8EF6F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 16:29:54,822 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:54,823 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:54,823 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:54,823 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: ED8345720D7C424EB8903558ABB8EF6F
2016-01-14 16:29:54,824 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 16:29:54,823 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 4001295E0A6D47EF95A734E1194DB0E4
2016-01-14 16:29:54,824 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 4001295E0A6D47EF95A734E1194DB0E4
2016-01-14 16:29:54,824 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: ED8345720D7C424EB8903558ABB8EF6F
2016-01-14 16:29:54,826 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 16:29:54,826 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 4001295E0A6D47EF95A734E1194DB0E4
2016-01-14 16:29:54,827 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 4001295E0A6D47EF95A734E1194DB0E4
2016-01-14 16:29:54,826 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:54,833 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EBC5870300144420851B41AA31EC272E, ordinal: 0
2016-01-14 16:29:54,833 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EBC5870300144420851B41AA31EC272E, mem on start (mb), free: 137, total: 357, max: 672
2016-01-14 16:29:54,834 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:54,834 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:54,834 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:54,835 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 16:29:54,835 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: ED8345720D7C424EB8903558ABB8EF6F
2016-01-14 16:29:54,835 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:54,836 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:54,836 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 16:29:54,836 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:54,836 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 16:29:54,836 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:54,836 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 16:29:54,837 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-01-14 16:29:54,837 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 4001295E0A6D47EF95A734E1194DB0E4, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 16:29:54,837 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, ED8345720D7C424EB8903558ABB8EF6F, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 16:29:54,837 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:54,837 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 4001295E0A6D47EF95A734E1194DB0E4 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:54,837 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 4001295E0A6D47EF95A734E1194DB0E4
2016-01-14 16:29:54,837 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:54,837 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 4001295E0A6D47EF95A734E1194DB0E4
2016-01-14 16:29:54,837 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 16:29:54,838 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:54,838 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:54,838 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788994620_0001_1_00_000000_0
2016-01-14 16:29:54,839 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 16:29:54,840 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@335bf8f0
2016-01-14 16:29:54,841 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@335bf8f0
2016-01-14 16:29:54,841 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 16:29:54,841 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 16:29:54,841 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 16:29:54,842 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 4001295E0A6D47EF95A734E1194DB0E4
2016-01-14 16:29:54,842 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter"], id: ED8345720D7C424EB8903558ABB8EF6F
2016-01-14 16:29:54,842 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter"]
2016-01-14 16:29:54,842 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 16:29:54,842 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:54,843 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EBC5870300144420851B41AA31EC272E, all 1 inputs ready in: 00:00:00.000
2016-01-14 16:29:54,844 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 16:29:54,845 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 16:29:54,845 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452788994620_0001_1_00_000000_0
2016-01-14 16:29:54,845 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452788994620_0001_1_00_000000_0 given a go for committing the task output.
2016-01-14 16:29:54,846 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788994620_0001_1_00_000000_0
2016-01-14 16:29:54,846 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EBC5870300144420851B41AA31EC272E, mem on close (mb), free: 135, total: 357, max: 672
2016-01-14 16:29:54 Completed running task attempt: attempt_1452788994620_0001_1_00_000000_0
2016-01-14 16:29:54,847 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788994620_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:54,850 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994620_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EBC5870300144420851B41AA31EC272E, taskAttemptId=attempt_1452788994620_0001_1_00_000000_0, startTime=1452788994815, finishTime=1452788994849, timeTaken=34, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=507, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374341632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1452788994843, Process_Duration=3, Process_End_Time=1452788994846, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3
2016-01-14 16:29:54,850 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788994620_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:54,850 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:54,850 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788994620_0001_1_00_000000_0
2016-01-14 16:29:54,851 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:54,851 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994620_0001_1][Event:TASK_FINISHED]: vertexName=EBC5870300144420851B41AA31EC272E, taskId=task_1452788994620_0001_1_00_000000, startTime=1452788994815, finishTime=1452788994850, timeTaken=35, status=SUCCEEDED, successfulAttemptID=attempt_1452788994620_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=507, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374341632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1452788994843, Process_Duration=3, Process_End_Time=1452788994846, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3
2016-01-14 16:29:54,851 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788994620_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:54,851 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788994620_0001_1_00 [EBC5870300144420851B41AA31EC272E] : 1
2016-01-14 16:29:54,851 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788994620_0001_1_00 [EBC5870300144420851B41AA31EC272E]
2016-01-14 16:29:54,852 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:54,852 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:54,852 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994620_0001_1][Event:VERTEX_FINISHED]: vertexName=EBC5870300144420851B41AA31EC272E, vertexId=vertex_1452788994620_0001_1_00, initRequestedTime=1452788994787, initedTime=1452788994801, startRequestedTime=1452788994802, startedTime=1452788994802, finishTime=1452788994851, timeTaken=49, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=507, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374341632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1452788994843, Process_Duration=3, Process_End_Time=1452788994846, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3, vertexStats=firstTaskStartTime=1452788994815, firstTasksToStart=[ task_1452788994620_0001_1_00_000000 ], lastTaskFinishTime=1452788994849, lastTasksToFinish=[ task_1452788994620_0001_1_00_000000 ], minTaskDuration=34, maxTaskDuration=34, avgTaskDuration=34.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788994620_0001_1_00_000000 ], longestDurationTasks=[ task_1452788994620_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:54,852 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788994620_0001_1_00 [EBC5870300144420851B41AA31EC272E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:54,853 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788994620_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:54,853 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994620_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788994620_0001_00_000001, stoppedTime=1452788994853, exitStatus=0
2016-01-14 16:29:54,853 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:54,853 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788994620_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:54,853 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788994620_0001_1_00 [EBC5870300144420851B41AA31EC272E] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-01-14 16:29:54,853 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452788994620_0001_1
2016-01-14 16:29:54,854 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994620_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452788994620_0001_1
2016-01-14 16:29:54,854 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: ED8345720D7C424EB8903558ABB8EF6F for vertex: vertex_1452788994620_0001_1_00 [EBC5870300144420851B41AA31EC272E]
2016-01-14 16:29:54,854 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=507, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374341632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3][cascading.flow.SliceCounters Process_Begin_Time=1452788994843, Process_Duration=3, Process_End_Time=1452788994846, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10, Tuples_Written=3]]
2016-01-14 16:29:54,855 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:54,855 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:54,855 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788994620_0001_00_000001
2016-01-14 16:29:54,855 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452788994620_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 16:29:54,856 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788994620_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 16:29:54,856 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452788994620_0001_00_000001, Exiting
2016-01-14 16:29:54,856 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788994620_0001_00_000001
2016-01-14 16:29:54,856 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788994620_0001_00_000001
2016-01-14 16:29:54,857 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788994620_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 16:29:54,857 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452788994620_0001_00_000001 completed successfully
2016-01-14 16:29:54,863 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452788994620_0001_1, queueSize=0
2016-01-14 16:29:54,863 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 16:29:54,863 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788994620_0001_1][Event:DAG_FINISHED]: dagId=dag_1452788994620_0001_1, startTime=1452788994785, finishTime=1452788994862, timeTaken=77, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=507, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374341632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1452788994843, Process_Duration=3, Process_End_Time=1452788994846, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3
2016-01-14 16:29:54,863 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452788994620_0001_1 finished with state: SUCCEEDED
2016-01-14 16:29:54,863 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788994620_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 16:29:54,866 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452788994620_0001_00_000001
2016-01-14 16:29:54,866 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788994620_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 16:29:54 Completed Dag: dag_1452788994620_0001_1
2016-01-14 16:29:54,866 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452788994620_0001_1, dagState=SUCCEEDED
2016-01-14 16:29:54,866 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788994620_0001_1_post
2016-01-14 16:29:54,866 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:54,867 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 16:29:54,869 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testFilter/filter, applicationId=application_1452788994620_0001
2016-01-14 16:29:54,871 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 16:29:54,871 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 16:29:54,871 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 16:29:54,871 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 16:29:54,871 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 16:29:54,871 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 16:29:54,872 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 16:29:54,873 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 16:29:54,873 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 16:29:54,873 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 16:29:54,873 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 16:29:54,873 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 16:29:54,874 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 16:29:54,877 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0687247E815F4851A88137658662BFF1/F2BECC5A70A4482A8AF418028EAA4065/.tez/application_1452788994620_0001
2016-01-14 16:29:54,877 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:00.275
2016-01-14 16:29:54,911 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testFilter/filter
2016-01-14 16:29:54,937 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 16:29:54,937 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 16:29:54,973 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-01-14 16:29:54,981 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-01-14 16:29:54,982 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 16:29:54,982 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 16:29:54,982 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 16:29:55,004 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-01-14 16:29:55,011 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple
2016-01-14 16:29:55,061 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 16:29:55,061 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 16:29:55,061 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple"]
2016-01-14 16:29:55,062 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 16:29:55,062 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 16:29:55,062 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 16:29:55,062 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...latform/testSimple/simple
2016-01-14 16:29:55,063 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 16:29:55,074 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 16:29:55,078 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:55,079 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 16:29:55,079 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 16:29:55,088 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/86D5671D48D34075B679018D0DE5D9F7/F24837A0DEA94251BD29064AD7E02892/.tez/application_1452788995079_0001 doesn't exist and is created
2016-01-14 16:29:55,104 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 16:29:55,104 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 16:29:55,105 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/86D5671D48D34075B679018D0DE5D9F7/F24837A0DEA94251BD29064AD7E02892/.tez/application_1452788995079_0001
2016-01-14 16:29:55,108 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452788995079_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 16:29:55,117 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 16:29:55,118 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 16:29:55,119 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 16:29:55,119 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 16:29:55,120 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/86D5671D48D34075B679018D0DE5D9F7/F24837A0DEA94251BD29064AD7E02892/.tez/application_1452788995079_0001/localmode-log-dir
2016-01-14 16:29:55,120 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/86D5671D48D34075B679018D0DE5D9F7/F24837A0DEA94251BD29064AD7E02892/.tez/application_1452788995079_0001/localmode-log-dir/history.txt.appattempt_1452788995079_0001_000000, maxErrors=10
2016-01-14 16:29:55,120 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 16:29:55,120 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452788995079_0001_000000, appSubmitTime=1452788995107, launchTime=1452788995107
2016-01-14 16:29:55,124 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 16:29:55,126 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 16:29:55,127 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 16:29:55,127 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 16:29:55,130 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:33485
2016-01-14 16:29:55,131 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 16:29:55,133 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452788995079_0001_000000, startTime=1452788995133
2016-01-14 16:29:55,133 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 16:29:55,133 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 16:29:55,205 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 16:29:55,205 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 16:29:55,205 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testSimple/simple, applicationId=application_1452788995079_0001, dagName=[86D5671D48D34075B679018D0DE5D9F7/F24837A0DEA94251BD29064AD7E02892] test/(1/1) ...latform/testSimple/simple
2016-01-14 16:29:55,220 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [86D5671D48D34075B679018D0DE5D9F7/F24837A0DEA94251BD29064AD7E02892] test/(1/1) ...latform/testSimple/simple
2016-01-14 16:29:55,221 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452788995079_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/86D5671D48D34075B679018D0DE5D9F7/F24837A0DEA94251BD29064AD7E02892/.tez/application_1452788995079_0001/localmode-log-dir/dag_1452788995079_0001_1.dot
2016-01-14 16:29:55,222 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/86D5671D48D34075B679018D0DE5D9F7/F24837A0DEA94251BD29064AD7E02892/.tez/application_1452788995079_0001/localmode-log-dir/dag_1452788995079_0001_1-tez-dag.pb.txt
2016-01-14 16:29:55,240 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788995079_0001_1
2016-01-14 16:29:55,240 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:55,240 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [86D5671D48D34075B679018D0DE5D9F7/F24837A0DEA94251BD29064AD7E02892] test/(1/1) ...latform/testSimple/simple
2016-01-14 16:29:55 Running Dag: dag_1452788995079_0001_1
2016-01-14 16:29:55,252 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 16:29:55,252 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788995079_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452788995079_0001_1, submitTime=1452788995220
2016-01-14 16:29:55,253 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E865346CD7B34B51BA4A9D0FD2F47774
2016-01-14 16:29:55,253 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 16:29:55,253 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 16:29:55,254 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788995079_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452788995079_0001_1, initTime=1452788995252
2016-01-14 16:29:55,254 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788995079_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 16:29:55,255 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788995079_0001_1][Event:DAG_STARTED]: dagID=dag_1452788995079_0001_1, startTime=1452788995254
2016-01-14 16:29:55,255 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 16:29:55,255 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788995079_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 16:29:55,255 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F35A3AB5EAEE431396493F7B28348957 : {213D67C179F64CE6810B9A38900F4269={InputName=213D67C179F64CE6810B9A38900F4269}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 16:29:55,255 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 213D67C179F64CE6810B9A38900F4269, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 16:29:55,255 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452788995079_0001_1_00 [F35A3AB5EAEE431396493F7B28348957]
2016-01-14 16:29:55,255 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452788995079_0001_1_00 [F35A3AB5EAEE431396493F7B28348957]
2016-01-14 16:29:55,256 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452788995079_0001_1_00 [F35A3AB5EAEE431396493F7B28348957]
2016-01-14 16:29:55,256 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452788995079_0001_1_00 [F35A3AB5EAEE431396493F7B28348957]. Starting root input initializers: 1
2016-01-14 16:29:55,255 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 16:29:55,256 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788995079_0001_1_00 [F35A3AB5EAEE431396493F7B28348957] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 16:29:55,257 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452788995079_0001_1_01 [E865346CD7B34B51BA4A9D0FD2F47774]
2016-01-14 16:29:55,257 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testSimple/simple, applicationId=application_1452788995079_0001, dagName=[86D5671D48D34075B679018D0DE5D9F7/F24837A0DEA94251BD29064AD7E02892] test/(1/1) ...latform/testSimple/simple
2016-01-14 16:29:55,257 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 213D67C179F64CE6810B9A38900F4269 on vertex vertex_1452788995079_0001_1_00 [F35A3AB5EAEE431396493F7B28348957]
2016-01-14 16:29:55,259 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 16:29:55,259 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 213D67C179F64CE6810B9A38900F4269 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 16:29:55,260 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 16:29:55,260 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452788995079_0001_1_01 [E865346CD7B34B51BA4A9D0FD2F47774]
2016-01-14 16:29:55,260 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 16:29:55,260 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452788995079_0001_1_01 [E865346CD7B34B51BA4A9D0FD2F47774]
2016-01-14 16:29:55,260 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 16:29:55,260 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452788995079_0001_1_01 [E865346CD7B34B51BA4A9D0FD2F47774]
2016-01-14 16:29:55,261 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=1402549C4C2641BCB194B028205C7922, vertexId=vertex_1452788995079_0001_1_01 [E865346CD7B34B51BA4A9D0FD2F47774], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 16:29:55,262 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=1402549C4C2641BCB194B028205C7922, vertexId=vertex_1452788995079_0001_1_01 [E865346CD7B34B51BA4A9D0FD2F47774]
2016-01-14 16:29:55,263 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-01-14 16:29:55,263 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 213D67C179F64CE6810B9A38900F4269 on vertex vertex_1452788995079_0001_1_00 [F35A3AB5EAEE431396493F7B28348957]
2016-01-14 16:29:55,264 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=1402549C4C2641BCB194B028205C7922, vertexName=E865346CD7B34B51BA4A9D0FD2F47774, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 16:29:55,264 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=1402549C4C2641BCB194B028205C7922, vertexId=vertex_1452788995079_0001_1_01 [E865346CD7B34B51BA4A9D0FD2F47774]
2016-01-14 16:29:55,266 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:55,266 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788995079_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E865346CD7B34B51BA4A9D0FD2F47774, vertexId=vertex_1452788995079_0001_1_01, initRequestedTime=1452788995257, initedTime=1452788995266, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 16:29:55,266 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788995079_0001_1_01 [E865346CD7B34B51BA4A9D0FD2F47774] transitioned from NEW to INITED due to event V_INIT
2016-01-14 16:29:55,267 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788995079_0001_1_00 [F35A3AB5EAEE431396493F7B28348957] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:55,267 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {213D67C179F64CE6810B9A38900F4269=forAllWorkUnits=true, update=[1]}
2016-01-14 16:29:55,267 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452788995079_0001_1_00 [F35A3AB5EAEE431396493F7B28348957] parallelism set to 1
2016-01-14 16:29:55,267 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452788995079_0001_1_00 [F35A3AB5EAEE431396493F7B28348957] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 16:29:55,268 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788995079_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788995079_0001_1_00, eventCount=1
2016-01-14 16:29:55,268 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788995079_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F35A3AB5EAEE431396493F7B28348957, vertexId=vertex_1452788995079_0001_1_00, initRequestedTime=1452788995255, initedTime=1452788995268, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 16:29:55,269 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788995079_0001_1_00 [F35A3AB5EAEE431396493F7B28348957] with distanceFromRoot: 0
2016-01-14 16:29:55,269 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788995079_0001_1_00 [F35A3AB5EAEE431396493F7B28348957] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 16:29:55,268 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 16:29:55,269 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F35A3AB5EAEE431396493F7B28348957
2016-01-14 16:29:55,269 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788995079_0001_1_00 [F35A3AB5EAEE431396493F7B28348957]
2016-01-14 16:29:55,270 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788995079_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788995079_0001_1_00, startRequestedTime=1452788995269, startedTime=1452788995269
2016-01-14 16:29:55,270 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:55,270 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788995079_0001_1_00 [F35A3AB5EAEE431396493F7B28348957] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:55,271 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:55,271 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788995079_0001_1][Event:TASK_STARTED]: vertexName=F35A3AB5EAEE431396493F7B28348957, taskId=task_1452788995079_0001_1_00_000000, scheduledTime=1452788995271, launchTime=1452788995271
2016-01-14 16:29:55,271 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788995079_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:55,271 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452788995079_0001_1_00 for vertex: vertex_1452788995079_0001_1_01 [E865346CD7B34B51BA4A9D0FD2F47774] numStartedSources: 1 numSources: 1
2016-01-14 16:29:55,272 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452788995079_0001_1_01 [E865346CD7B34B51BA4A9D0FD2F47774] with distanceFromRoot: 1
2016-01-14 16:29:55,272 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788995079_0001_1_00 [F35A3AB5EAEE431396493F7B28348957]
2016-01-14 16:29:55,272 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F35A3AB5EAEE431396493F7B28348957 in vertex: E865346CD7B34B51BA4A9D0FD2F47774
2016-01-14 16:29:55,272 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E865346CD7B34B51BA4A9D0FD2F47774 with 1 source tasks and 1 pending tasks
2016-01-14 16:29:55,273 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788995079_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452788995079_0001_1_01, startRequestedTime=1452788995272, startedTime=1452788995272
2016-01-14 16:29:55,273 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 16:29:55,273 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788995079_0001_1_01 [E865346CD7B34B51BA4A9D0FD2F47774] transitioned from INITED to RUNNING due to event V_START
2016-01-14 16:29:55,273 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452788995079_0001, with dag id: dag_1452788995079_0001_1
2016-01-14 16:29:55,274 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [86D5671D48D34075B679018D0DE5D9F7/F24837A0DEA94251BD29064AD7E02892] test/(1/1) ...latform/testSimple/simple, VertexName: F35A3AB5EAEE431396493F7B28348957, VertexParallelism: 1, TaskAttemptID:attempt_1452788995079_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=213D67C179F64CE6810B9A38900F4269, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E865346CD7B34B51BA4A9D0FD2F47774, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 16:29:55,274 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788995079_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:55,274 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:55,276 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 16:29:55,277 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 16:29:55,277 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 16:29:55,277 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788995079_0001_00_000001
2016-01-14 16:29:55,277 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788995079_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:55,279 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788995079_0001_00_000001
2016-01-14 16:29:55,279 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 16:29:55,283 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788995079_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:55,283 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788995079_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788995079_0001_00_000001, launchTime=1452788995283
2016-01-14 16:29:55,284 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788995079_0001_00_000001
2016-01-14 16:29:55,284 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:55,285 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788995079_0001_1_00_000000_0] to container: [container_1452788995079_0001_00_000001]
2016-01-14 16:29:55,285 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788995079_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:55,286 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788995079_0001_1_00_000000_0] started. Is using containerId: [container_1452788995079_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 16:29:55,286 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788995079_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F35A3AB5EAEE431396493F7B28348957, taskAttemptId=attempt_1452788995079_0001_1_00_000000_0, startTime=1452788995286, containerId=container_1452788995079_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788995079_0001_00_000001/teamcity, completedLogs=
2016-01-14 16:29:55,286 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:55,286 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788995079_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:55,286 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788995079_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:55,287 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788995079_0001_00_000001 given task: attempt_1452788995079_0001_1_00_000000_0
2016-01-14 16:29:55,288 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788995079_0001_00_000001: 1 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788995079_0001_1_00_000000_0
2016-01-14 16:29:55 Starting to run new task attempt: attempt_1452788995079_0001_1_00_000000_0
2016-01-14 16:29:55,288 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788995079_0001_1_00_000000_0
2016-01-14 16:29:55,288 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:55,288 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:55,289 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:55,289 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:55,289 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:55,289 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:55,289 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [86D5671D48D34075B679018D0DE5D9F7/F24837A0DEA94251BD29064AD7E02892] test/(1/1) ...latform/testSimple/simple, VertexName: F35A3AB5EAEE431396493F7B28348957, VertexParallelism: 1, TaskAttemptID:attempt_1452788995079_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=213D67C179F64CE6810B9A38900F4269, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E865346CD7B34B51BA4A9D0FD2F47774, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 16:29:55,289 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:55,290 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788995079_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 16:29:55,290 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788995079_0001_1_00_000000_0
2016-01-14 16:29:55,291 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:55,291 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:55,291 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=213D67C179F64CE6810B9A38900F4269, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 16:29:55,292 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E865346CD7B34B51BA4A9D0FD2F47774, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 16:29:55,291 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:55,293 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:55,293 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E865346CD7B34B51BA4A9D0FD2F47774
2016-01-14 16:29:55,295 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 16:29:55,296 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E865346CD7B34B51BA4A9D0FD2F47774
2016-01-14 16:29:55,297 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E865346CD7B34B51BA4A9D0FD2F47774
2016-01-14 16:29:55,292 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:55,297 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 16:29:55,298 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 16:29:55,295 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:55,298 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 213D67C179F64CE6810B9A38900F4269
2016-01-14 16:29:55,301 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 213D67C179F64CE6810B9A38900F4269
2016-01-14 16:29:55,304 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 16:29:55,304 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 213D67C179F64CE6810B9A38900F4269
2016-01-14 16:29:55,305 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 213D67C179F64CE6810B9A38900F4269
2016-01-14 16:29:55,307 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F35A3AB5EAEE431396493F7B28348957, ordinal: 0
2016-01-14 16:29:55,307 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F35A3AB5EAEE431396493F7B28348957, mem on start (mb), free: 265, total: 358, max: 672
2016-01-14 16:29:55,308 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:55,308 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:55,308 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:55,308 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:55,309 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:55,309 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 16:29:55,309 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 16:29:55,310 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:55,310 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 16:29:55,310 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 16:29:55,310 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E865346CD7B34B51BA4A9D0FD2F47774, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 16:29:55,311 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 213D67C179F64CE6810B9A38900F4269, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 16:29:55,311 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:55,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 213D67C179F64CE6810B9A38900F4269
2016-01-14 16:29:55,312 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:55,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 213D67C179F64CE6810B9A38900F4269
2016-01-14 16:29:55,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 213D67C179F64CE6810B9A38900F4269 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:55,313 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 16:29:55,313 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:55,314 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:55,314 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788995079_0001_1_00_000000_0
2016-01-14 16:29:55,314 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 16:29:55,315 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2199f75a
2016-01-14 16:29:55,316 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2199f75a
2016-01-14 16:29:55,316 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 16:29:55,317 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 16:29:55,318 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 16:29:55,319 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 213D67C179F64CE6810B9A38900F4269
2016-01-14 16:29:55,319 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: 8AEEB643BD1F48EDA4B2E7F2DAAF90A7
2016-01-14 16:29:55,320 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] 8AEEB643BD1F48EDA4B2E7F2DAAF90A7
2016-01-14 16:29:55,320 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@2cf68b72; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@6e3d006d; comparator=cascading.tuple.hadoop.util.TupleComparator@66e72ca5; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 16:29:55,321 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 16:29:55,321 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 16:29:55,321 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 16:29:55,334 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 16:29:55,334 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 16:29:55,334 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 16:29:55,334 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 16:29:55,335 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 16:29:55,335 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 16:29:55,336 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 16:29:55,336 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F35A3AB5EAEE431396493F7B28348957, all 1 inputs ready in: 00:00:00.000
2016-01-14 16:29:55,340 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788995079_0001_1_00_000000_0
2016-01-14 16:29:55,340 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F35A3AB5EAEE431396493F7B28348957, mem on close (mb), free: 163, total: 358, max: 672
2016-01-14 16:29:55,340 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 16:29:55,340 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 16:29:55,340 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 210; bufvoid = 104857600
2016-01-14 16:29:55,341 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214360(104857440); length = 37/6553600
2016-01-14 16:29:55,344 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 16:29:55 Completed running task attempt: attempt_1452788995079_0001_1_00_000000_0
2016-01-14 16:29:55,347 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788995079_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:55,347 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788995079_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452788995079_0001_1_00, eventCount=1
2016-01-14 16:29:55,347 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375390208, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452788995336, Process_Duration=4, Process_End_Time=1452788995340, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10]]
2016-01-14 16:29:55,348 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:55,348 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:55,348 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788995079_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F35A3AB5EAEE431396493F7B28348957, taskAttemptId=attempt_1452788995079_0001_1_00_000000_0, startTime=1452788995286, finishTime=1452788995348, timeTaken=62, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375390208, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788995336, Process_Duration=4, Process_End_Time=1452788995340, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-01-14 16:29:55,348 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788995079_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:55,348 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:55,348 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:55,348 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788995079_0001_1_00_000000_0
2016-01-14 16:29:55,349 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788995079_0001_00_000001
2016-01-14 16:29:55,350 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:55,350 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788995079_0001_1][Event:TASK_FINISHED]: vertexName=F35A3AB5EAEE431396493F7B28348957, taskId=task_1452788995079_0001_1_00_000000, startTime=1452788995286, finishTime=1452788995349, timeTaken=63, status=SUCCEEDED, successfulAttemptID=attempt_1452788995079_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375390208, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788995336, Process_Duration=4, Process_End_Time=1452788995340, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-01-14 16:29:55,350 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788995079_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:55,350 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788995079_0001_1_00 [F35A3AB5EAEE431396493F7B28348957] : 1
2016-01-14 16:29:55,350 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788995079_0001_1_00 [F35A3AB5EAEE431396493F7B28348957]
2016-01-14 16:29:55,351 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:55,351 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788995079_0001_1][Event:VERTEX_FINISHED]: vertexName=F35A3AB5EAEE431396493F7B28348957, vertexId=vertex_1452788995079_0001_1_00, initRequestedTime=1452788995255, initedTime=1452788995268, startRequestedTime=1452788995269, startedTime=1452788995269, finishTime=1452788995350, timeTaken=81, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375390208, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788995336, Process_Duration=4, Process_End_Time=1452788995340, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1452788995286, firstTasksToStart=[ task_1452788995079_0001_1_00_000000 ], lastTaskFinishTime=1452788995348, lastTasksToFinish=[ task_1452788995079_0001_1_00_000000 ], minTaskDuration=62, maxTaskDuration=62, avgTaskDuration=62.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788995079_0001_1_00_000000 ], longestDurationTasks=[ task_1452788995079_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:55,351 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788995079_0001_1_00 [F35A3AB5EAEE431396493F7B28348957] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:55,352 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788995079_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:55,352 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788995079_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788995079_0001_00_000001, stoppedTime=1452788995352, exitStatus=0
2016-01-14 16:29:55,352 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:55,352 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788995079_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:55,352 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452788995079_0001_1_01 [E865346CD7B34B51BA4A9D0FD2F47774] attempt: attempt_1452788995079_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 16:29:55,352 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E865346CD7B34B51BA4A9D0FD2F47774
2016-01-14 16:29:55,352 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452788995079_0001_1_01 [E865346CD7B34B51BA4A9D0FD2F47774]
2016-01-14 16:29:55,352 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788995079_0001_1_00 [F35A3AB5EAEE431396493F7B28348957] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 16:29:55,353 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452788995079_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 16:29:55,353 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788995079_0001_1][Event:TASK_STARTED]: vertexName=E865346CD7B34B51BA4A9D0FD2F47774, taskId=task_1452788995079_0001_1_01_000000, scheduledTime=1452788995353, launchTime=1452788995353
2016-01-14 16:29:55,353 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 16:29:55,353 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788995079_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 16:29:55,353 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788995079_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 16:29:55,353 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452788995079_0001_1_01 [E865346CD7B34B51BA4A9D0FD2F47774]
2016-01-14 16:29:55,353 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [86D5671D48D34075B679018D0DE5D9F7/F24837A0DEA94251BD29064AD7E02892] test/(1/1) ...latform/testSimple/simple, VertexName: E865346CD7B34B51BA4A9D0FD2F47774, VertexParallelism: 1, TaskAttemptID:attempt_1452788995079_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F35A3AB5EAEE431396493F7B28348957, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=1402549C4C2641BCB194B028205C7922, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:55,354 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788995079_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 16:29:55,354 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 16:29:55,355 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452788995079_0001_00_000002
2016-01-14 16:29:55,355 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788995079_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 16:29:55,357 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452788995079_0001_00_000002
2016-01-14 16:29:55,357 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-01-14 16:29:55,357 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788995079_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 16:29:55,358 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 16:29:55,358 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788995079_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452788995079_0001_00_000002, launchTime=1452788995357
2016-01-14 16:29:55,358 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788995079_0001_00_000002
2016-01-14 16:29:55,359 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452788995079_0001_1_01_000000_0] to container: [container_1452788995079_0001_00_000002]
2016-01-14 16:29:55,359 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788995079_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 16:29:55,359 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452788995079_0001_1_01_000000_0] started. Is using containerId: [container_1452788995079_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 16:29:55,359 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788995079_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E865346CD7B34B51BA4A9D0FD2F47774, taskAttemptId=attempt_1452788995079_0001_1_01_000000_0, startTime=1452788995359, containerId=container_1452788995079_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452788995079_0001_00_000002/teamcity, completedLogs=
2016-01-14 16:29:55,359 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 16:29:55,360 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452788995079_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 16:29:55,360 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788995079_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 16:29:55,359 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788995079_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 16:29:55,360 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452788995079_0001_00_000001, Exiting
2016-01-14 16:29:55,360 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452788995079_0001_00_000002 given task: attempt_1452788995079_0001_1_01_000000_0
2016-01-14 16:29:55,360 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788995079_0001_00_000001
2016-01-14 16:29:55,360 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788995079_0001_00_000001
2016-01-14 16:29:55,360 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788995079_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 16:29:55,361 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788995079_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452788995079_0001_1_01_000000_0
2016-01-14 16:29:55,361 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788995079_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 16:29:55,361 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452788995079_0001_00_000001 completed successfully
2016-01-14 16:29:55 Starting to run new task attempt: attempt_1452788995079_0001_1_01_000000_0
2016-01-14 16:29:55,361 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788995079_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 16:29:55,361 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452788995079_0001_1_01_000000_0
2016-01-14 16:29:55,362 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:55,362 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 16:29:55,362 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 16:29:55,362 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 16:29:55,362 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 16:29:55,362 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 16:29:55,363 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [86D5671D48D34075B679018D0DE5D9F7/F24837A0DEA94251BD29064AD7E02892] test/(1/1) ...latform/testSimple/simple, VertexName: E865346CD7B34B51BA4A9D0FD2F47774, VertexParallelism: 1, TaskAttemptID:attempt_1452788995079_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F35A3AB5EAEE431396493F7B28348957, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=1402549C4C2641BCB194B028205C7922, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 16:29:55,363 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 16:29:55,363 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452788995079_0001_1_01_000000_0
2016-01-14 16:29:55,364 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 16:29:55,364 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:55,364 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452788995079_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 16:29:55,364 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F35A3AB5EAEE431396493F7B28348957, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 16:29:55,365 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=1402549C4C2641BCB194B028205C7922, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 16:29:55,366 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 16:29:55,367 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 1402549C4C2641BCB194B028205C7922
2016-01-14 16:29:55,367 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 16:29:55,367 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 1402549C4C2641BCB194B028205C7922
2016-01-14 16:29:55,365 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 16:29:55,368 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F35A3AB5EAEE431396493F7B28348957
2016-01-14 16:29:55,364 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:55,370 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 16:29:55,370 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 16:29:55,370 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: F35A3AB5EAEE431396493F7B28348957
2016-01-14 16:29:55,372 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F35A3AB5EAEE431396493F7B28348957
2016-01-14 16:29:55,373 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 16:29:55,379 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 16:29:55,379 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 1402549C4C2641BCB194B028205C7922
2016-01-14 16:29:55,383 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E865346CD7B34B51BA4A9D0FD2F47774, ordinal: 1
2016-01-14 16:29:55,383 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E865346CD7B34B51BA4A9D0FD2F47774, mem on start (mb), free: 158, total: 358, max: 672
2016-01-14 16:29:55,383 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 16:29:55,384 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 16:29:55,384 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 16:29:55,384 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 16:29:55,384 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 16:29:55,385 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 16:29:55,385 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 16:29:55,385 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 16:29:55,386 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 16:29:55,386 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 16:29:55,386 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 1402549C4C2641BCB194B028205C7922, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 16:29:55,387 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F35A3AB5EAEE431396493F7B28348957, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 16:29:55,387 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 16:29:55,387 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F35A3AB5EAEE431396493F7B28348957 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 16:29:55,387 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F35A3AB5EAEE431396493F7B28348957
2016-01-14 16:29:55,388 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 16:29:55,388 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 16:29:55,388 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 16:29:55,389 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 16:29:55,389 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F35A3AB5EAEE431396493F7B28348957 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 16:29:55,389 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 16:29:55,389 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 16:29:55,389 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 16:29:55,399 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F35A3AB5EAEE431396493F7B28348957
2016-01-14 16:29:55,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 16:29:55,400 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452788995079_0001_1_01_000000_0
2016-01-14 16:29:55,405 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-01-14 16:29:55,406 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452788995079_0001_1_00_000000_0_10089, runDuration: 52007, ]
2016-01-14 16:29:55,406 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: 8AEEB643BD1F48EDA4B2E7F2DAAF90A7
2016-01-14 16:29:55,407 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple"], id: 1402549C4C2641BCB194B028205C7922
2016-01-14 16:29:55,407 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple"]
2016-01-14 16:29:55,407 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] 8AEEB643BD1F48EDA4B2E7F2DAAF90A7, for 1 inputs
2016-01-14 16:29:55,408 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452788995079_0001_1_00_000000_0_10089] to DISK_DIRECT, CompressedSize=204, DecompressedSize=200,EndTime=1452788995408, TimeTaken=1, Rate=0.19 MB/s
2016-01-14 16:29:55,408 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F35A3AB5EAEE431396493F7B28348957
2016-01-14 16:29:55,409 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 16:29:55,409 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [F35A3AB5EAEE431396493F7B28348957] #14 in 2ms
2016-01-14 16:29:55,409 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F35A3AB5EAEE431396493F7B28348957] #14, status:false, isInterrupted:false
2016-01-14 16:29:55,410 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 16:29:55,411 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 204 bytes from disk
2016-01-14 16:29:55,411 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 16:29:55,411 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 16:29:55,411 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 175 bytes
2016-01-14 16:29:55,412 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E865346CD7B34B51BA4A9D0FD2F47774, all 1 inputs ready in: 00:00:00.004
2016-01-14 16:29:55,412 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F35A3AB5EAEE431396493F7B28348957
2016-01-14 16:29:55,413 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@173dd131; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@5b70f3e2
2016-01-14 16:29:55,413 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 16:29:55,415 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 16:29:55,416 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 16:29:55,416 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452788995079_0001_1_01_000000_0
2016-01-14 16:29:55,416 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452788995079_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 16:29:55,417 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452788995079_0001_1_01_000000_0
2016-01-14 16:29:55,417 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E865346CD7B34B51BA4A9D0FD2F47774, mem on close (mb), free: 155, total: 358, max: 672
2016-01-14 16:29:55,418 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F35A3AB5EAEE431396493F7B28348957
2016-01-14 16:29:55,418 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 16:29:55,419 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 16:29:55,419 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 16:29:55,419 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 16:29:55,419 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 16:29:55 Completed running task attempt: attempt_1452788995079_0001_1_01_000000_0
2016-01-14 16:29:55,420 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452788995079_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 16:29:55,421 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375390208, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=20, MERGE_PHASE_TIME=23, FIRST_EVENT_RECEIVED=18, LAST_EVENT_RECEIVED=18][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452788995412, Process_Duration=5, Process_End_Time=1452788995417, Read_Duration=0, Tuples_Read=10, Tuples_Written=8, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=8]]
2016-01-14 16:29:55,422 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788995079_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E865346CD7B34B51BA4A9D0FD2F47774, taskAttemptId=attempt_1452788995079_0001_1_01_000000_0, startTime=1452788995359, finishTime=1452788995421, timeTaken=62, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375390208, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=20, MERGE_PHASE_TIME=23, FIRST_EVENT_RECEIVED=18, LAST_EVENT_RECEIVED=18, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788995412, Process_Duration=5, Process_End_Time=1452788995417, Read_Duration=0, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-01-14 16:29:55,422 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 16:29:55,422 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452788995079_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 16:29:55,423 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 16:29:55,422 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 16:29:55,423 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:55,423 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452788995079_0001_1_01_000000_0
2016-01-14 16:29:55,423 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 16:29:55,424 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788995079_0001_1][Event:TASK_FINISHED]: vertexName=E865346CD7B34B51BA4A9D0FD2F47774, taskId=task_1452788995079_0001_1_01_000000, startTime=1452788995359, finishTime=1452788995423, timeTaken=64, status=SUCCEEDED, successfulAttemptID=attempt_1452788995079_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375390208, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=20, MERGE_PHASE_TIME=23, FIRST_EVENT_RECEIVED=18, LAST_EVENT_RECEIVED=18, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788995412, Process_Duration=5, Process_End_Time=1452788995417, Read_Duration=0, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-01-14 16:29:55,423 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452788995079_0001_00_000002
2016-01-14 16:29:55,424 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452788995079_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 16:29:55,424 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452788995079_0001_1_01 [E865346CD7B34B51BA4A9D0FD2F47774] : 1
2016-01-14 16:29:55,424 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452788995079_0001_1_01 [E865346CD7B34B51BA4A9D0FD2F47774]
2016-01-14 16:29:55,425 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 16:29:55,426 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788995079_0001_1][Event:VERTEX_FINISHED]: vertexName=E865346CD7B34B51BA4A9D0FD2F47774, vertexId=vertex_1452788995079_0001_1_01, initRequestedTime=1452788995257, initedTime=1452788995266, startRequestedTime=1452788995272, startedTime=1452788995272, finishTime=1452788995425, timeTaken=153, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375390208, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=20, MERGE_PHASE_TIME=23, FIRST_EVENT_RECEIVED=18, LAST_EVENT_RECEIVED=18, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452788995412, Process_Duration=5, Process_End_Time=1452788995417, Read_Duration=0, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8, vertexStats=firstTaskStartTime=1452788995359, firstTasksToStart=[ task_1452788995079_0001_1_01_000000 ], lastTaskFinishTime=1452788995421, lastTasksToFinish=[ task_1452788995079_0001_1_01_000000 ], minTaskDuration=62, maxTaskDuration=62, avgTaskDuration=62.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452788995079_0001_1_01_000000 ], longestDurationTasks=[ task_1452788995079_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 16:29:55,426 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452788995079_0001_1_01 [E865346CD7B34B51BA4A9D0FD2F47774] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 16:29:55,426 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788995079_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 16:29:55,426 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788995079_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452788995079_0001_00_000002, stoppedTime=1452788995426, exitStatus=0
2016-01-14 16:29:55,426 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 16:29:55,426 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788995079_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 16:29:55,427 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452788995079_0001_1_01 [E865346CD7B34B51BA4A9D0FD2F47774] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 16:29:55,427 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452788995079_0001_1
2016-01-14 16:29:55,427 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788995079_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452788995079_0001_1
2016-01-14 16:29:55,427 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452788995079_0001_1_00 [F35A3AB5EAEE431396493F7B28348957]
2016-01-14 16:29:55,427 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 1402549C4C2641BCB194B028205C7922 for vertex: vertex_1452788995079_0001_1_01 [E865346CD7B34B51BA4A9D0FD2F47774]
2016-01-14 16:29:55,434 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452788995079_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 16:29:55,435 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452788995079_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 16:29:55,435 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452788995079_0001_00_000002, Exiting
2016-01-14 16:29:55,435 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452788995079_0001_00_000002
2016-01-14 16:29:55,435 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452788995079_0001_00_000002
2016-01-14 16:29:55,435 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452788995079_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 16:29:55,435 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452788995079_0001_00_000002 completed successfully
2016-01-14 16:29:55,438 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452788995079_0001_1, queueSize=0
2016-01-14 16:29:55,438 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 16:29:55,439 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452788995079_0001_1][Event:DAG_FINISHED]: dagId=dag_1452788995079_0001_1, startTime=1452788995254, finishTime=1452788995436, timeTaken=182, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2442, FILE_BYTES_WRITTEN=376, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=750780416, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=18, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=20, MERGE_PHASE_TIME=23, FIRST_EVENT_RECEIVED=18, LAST_EVENT_RECEIVED=18, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905577990748, Process_Duration=9, Process_End_Time=2905577990757, Read_Duration=2, Tuples_Read=20, Tuples_Written=18, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=8
2016-01-14 16:29:55,439 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452788995079_0001_1 finished with state: SUCCEEDED
2016-01-14 16:29:55,439 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452788995079_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 16:29:55,440 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452788995079_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 16:29:55,440 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452788995079_0001_00_000002
2016-01-14 16:29:55 Completed Dag: dag_1452788995079_0001_1
2016-01-14 16:29:55,441 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452788995079_0001_1, dagState=SUCCEEDED
2016-01-14 16:29:55,441 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452788995079_0001_1_post
2016-01-14 16:29:55,441 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 16:29:55,441 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 16:29:55,447 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testSimple/simple, applicationId=application_1452788995079_0001
2016-01-14 16:29:55,448 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 16:29:55,448 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 16:29:55,448 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 16:29:55,448 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 16:29:55,448 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 16:29:55,449 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 16:29:55,449 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 16:29:55,449 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 16:29:55,450 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 16:29:55,450 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 16:29:55,451 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 16:29:55,451 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 16:29:55,451 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 16:29:55,457 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/86D5671D48D34075B679018D0DE5D9F7/F24837A0DEA94251BD29064AD7E02892/.tez/application_1452788995079_0001
2016-01-14 16:29:55,457 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:00.396
2016-01-14 16:29:55,489 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimple/simple